As I've commented on the board previously, I think we have an exaggerated idea of just how frequently the Edwardian rich changed their outfits throughout the day. Obviously, turn-of-the-century clothing was far more formal and far more complicated - all those buttons and hooks which in themselves required either great dexterity or the services of a maid or valet - but I don't believe that first-class passengers returned to their staterooms five or six times daily to change. What would be the point? Aboard ship, the field of social activity was far more restricted than on land and, besides, I have a feeling that any lady who appeared hourly in a brand-new Parisian frock would have been deemed flashy and vulgar by her peers.
But to return to your point, Vitezslav. Afternoon tea took place around 4.00PM or 5.00PM and (I seem to recall) was originally instituted by the Duchess of Bedford in 1840 as a means of filling the gap between lunch and dinner and also as a convenient way of entertaining callers one didn't necessarily want to provide a twelve-course banquet for. As we all know, Lucile specialised in loose and frothy tea-gowns, worn sans-corset, which provided ladies with a little light relief from the discomforts imposed by whale-bone and starch. But - and here I may be wrong - I suspect tea-gowns were more usually worn 'at home' rather than in the public arena - which the first-class lounge, reception room, palm courts and Cafe Parisien very much were. Taking tea aboard the 'Titanic' would have been akin to a rendezvous at the Ritz in London or the Plaza in New York. Flirty little scraps of chiffon and lace, worn in full view of three-hundred fellow travellers, would not have been deemed appropriate at such a time and in such a place. Moreover, falling as it did in the mid- to late-afternoon, teatime often caught both ladies and gentlemen 'on the hop' as they went about their daily business. During the Season, an impromptu gathering might unfold at any one of several thousand fashionable town-houses, with a motley array of visitors dropping by for a cup of Assam, a cucumber sandwich and other light but much-needed refreshments. Some gentleman might conceivably be en route to or from Westminster or the Palace and, if so, would have indeed worn formal sponge-bag trousers and cut-away coats. On the other hand, there might also be dapper chaps - those with no other demands on their time - sporting casual, light-weight summer suits and panama hats. And, if tea was taken on the manicured lawn or terrace of some country-house (a la Merchant-Ivory's 'Room with a View'), then visitors might be just as likely to sit down in tennis whites!
In other words, I doubt there were hard-and-fast rules which dictated what should or should not be worn at tea-time. It very much depended on the formality (or otherwise) of the venue.