Please also note, that for bmsr I am also getting the value 1111111111111111 from time to time. How can i change the clocks to be sure that it is the hardware problem, not software?
If you get corrupted data from the PHY then it could be a problem of unstable electrical connection. The driver reads the corrupted value and does the wrong thing.
BTW, one of the ChibiOS customers created a control board based on the STM32F4-Discovery, it has the DP83848 in RMII mode clocked by a MCO. I should receive a sample soon and will be able to answer the question.
The only problem I see is that in order to generate 50MHz the MCU clock must be set to 150MHz.
Main is 168MHz. Maybe this is the reason of my problem?
You mean 25cm traces from STM32 to PHY or from PHY to ethernet transformer (usually integrated into the RJ45 connector) ?.
In either case, that may very well be your problem. You must be very careful with traces from PHY to transformer, and in fact you should be using an impedance controlled differential pair of traces and be careful not to cross ground planes and stuff like that. In the case of traces from STM32 to PHY, length matching could help, and some series resistor damping too. If this is the case, try placing a series 33 ohm resistor near each output signal pin.
Ethernet signals are much more tricky than MII/RMII, so if you cannot shorten the distance, always place the PHY as close as possible to the ethernet transformer and focus in MII/RMII signal integrity.
