On one occasion, the Buddha said to the monks this:
"No other form do I know, O monks, that so persists in obsessing the mind of a man as the form of a woman. The form of a woman persists in obsessing the mind of a man.
You can sit at some time in the early morning if your working schedule allows you to do so. In the early morning it is often less noisy than other times during the day and the atmosphere is still more pure than later in the day or in the evening.
You may sit in the evening about 9:00 P.M. or later. It would be better if you start to sit about two hours after your meal because if your stomach is still full it will be not good for sitting. What has just been said is for a beginner, and if your are an enthusiast practitioner, whatever time would be good for you.