I think it makes sense. He doesn’t really recall the funnel collapsing because the heavy list made the event quick. In Cameron’s movie, the funnel sort of pauses for a moment once the guy wires snap, as there is no list. But in reality, the port list had been making the funnel want to fall over in that direction for a while.
The officers ordered everyone to the starboard side "to keep the ship up as long as possible" - Hemming. Lightoller said "When the order was given to the passengers to go to the starboard side I am under the impression that a great many went over and the ship got a righting movement and maintained it". Jack Thayer said - "She gradually came out of her list to port, and if anything, had a slight list to starboard." Frank Prentice was on the poop deck and said she appeared to "right herself". I believe the ship was twisting herself apart in opposite directions. Her stern wanted to lean over to port and her bow wanted to lean over to starboard. It is unknown where exactly the water was rushing to inside the ship e.g. via open portholes and fractures up and down the length of the ship and what role they played.
I believe the bow rolled over to starboard and threw the first two funnels over that way. The wreckage appears to show the starboard bridge wing and captain's quarters have been crushed by something large falling over that way.