If any, what would be the ideal modulation frequency for a phase-sensitive light sensing experiment in a non-dark environment?

Do you have access to a dynamic signal analyzer or similar? My most recent setup involves locking in to a signal from a split photodiode that is either singly or doubly modulated. We chose our base modulation frequency by looking at the noise spectrum -- our photodiode when illuminated with a DC signal looks something like this:

There's a ton of low-frequency noise including 60 Hz main noise and few-Hz from mechanical vibrations in our system. If the fluorescent lights are on we sometimes see signals at strange frequencies from those. The way we are doing our modulation scales poorly beyond about 10 kHz, so that was an upper limit. Because of this, we run our signal through a high-pass preamp with 12 dB/octave falloff. Then the signal response looks like

and we modulate at around 5 kHz which is safely above the high-pass but not so high that we lose modulation power. This might be overkill for your situation however, it's hard to say.

The psd method basically moves the signal from d.c. to some other frequency, and then is sensitive to noise at that other frequency. Therefore you pick a frequency where the noise is small. Many noise sources decrease with frequency, so often one picks as high a frequency as your detector and de-modulation stage (usually a lock-in amplifier) can handle.

From your description, it seems that your detection system is operating in a nonlinear region. Nonlinearities in the detection system will complicate things, so making sure the detector is operating in its linear range will be very important. If the detection is linear, separation of the ambient and signal beam effects should be fairly straightforward. A useful approach can be to directly measure the ambient light that leaks into your box. You can then find a way to separate the effects of your light beam and the ambient light.