The black Victorians had a rough time. It was generally not much different than that of the African Americans of this time period except that they were not entirely viewed as property rather than human beings.
However, since many of the black people of the Victorian era in England were immigrants, many of them took positions as nannies. These positions typically consisted of a lot of labor, but it was a job they were willing to do. They would do this work to get away from severe poverty in their native countries. In the end, the nanny positions would give them a place to live and things to eat. The wages, however, were minimal.
It was probably similar to being in prison. You were probably told what to do and when to do it, the consequence being though that you would be out on the dangerous streets of London if you did not do as you were told.
Those that did not have these types of positions lived in severe poverty. They were generally outcasts and had very few places to live or eat. Their employment opportunities were minimal simply because there were so many white people looking for jobs; it was almost always the black people who were stuck outside trying to find a place to put food on the table for their families.
Many of the black people of England in the Victorian era took very dangerous jobs. These jobs may have consisted of countryside mining positions where many would die at a very early age.
These are positions that most white people would not take simply because of what type of work it was. However, the black people in the Victorian era made many sacrifices and had very rough lives for several years.