Remote sensing images provide the capability to obtain information of various land cover classes. This information is useful to the process of urban planning, socio-economic modelling and population studies. There is a need for accurate and efficient methods to obtain this information, particularly in regions where there is a scarcity of reference data. In this work, we develop a methodology based on fully convolutional networks (FCN) that is trained in an end-to-end fashion using aerial RGB images only as input. The experiments are conducted on the city of Goma in the Democratic Republic of Congo. We compare the results to a state-of-the-art approach based on a semi-automatic Geographic object image-based analysis (GEOBIA) processing chain. State-of-the-art classification accuracies are obtained by both methods whereby FCN and the best baseline method have an overall accuracy of 91.24% and 89.34% respectively. The maps have good visual quality and the use of a FCN skip architecture minimizes the rounded edges that is characteristic of FCN maps. Finally, additional experiments are done to refine FCN classified maps using segments obtained from GEOBIA. This resulted in improved edge delineation in the FCN maps, and future work will involve explicitly incorporating boundary information from the GEOBIA segmentation into the FCN pipeline in an end-to-end fashion.