The only relevant passage is 1 Cor 11:4-7 containing some comments from Paul:
Every man who prays or prophesies with his head covered dishonors his head, but every wife who prays or prophesies with her head uncovered dishonors her head, since it is the same as if her head were shaven. For if a wife will not cover her head, then she should cut her hair short. But since it is disgraceful for a wife to cut off her hair or shave her head, let her cover her head. For a man ought not to cover his head, since he is the image and glory of God, but woman is the glory of man.
This is a rather famous and contentious passage with many different conflicting interpretations. Does Paul mean “wear a hat” by “cover”? His language is hardly unambiguous — for instance, he says “A woman’s hair is given to her for a covering.” (1 Cor 11:9). And Paul is somewhat unclear about what praying or prophesying is — is it listening/sharing, or leading? And it’s not exactly clear quite what the scope of Paul’s instruction here is — is it a one off resolution of a local cultural issue, or a general rule for all churches?
Jewish custom is to cover the head at all times, but it appears that firm Talmudic rulings on this are fairly recent (at least post-middle-age persecution), aimed at differentiating the Jews from the Christians, so it seems that this is no useful source of guidance on the cultural background to Paul’s words.
My own interpretation is that while Paul’s words are ambiguous, there is no harm in a man consistently having his head uncovered (no hat, no long hair) in a worship service where prayer will occur. Even when it’s a little cold.