Africa is estimated to have great potential for agricultural production, but there are a number of constraints inhibiting the development of that potential. Spatial data are increasingly important in the realization of potential as well as the associated constraints. With crop production data generated at 5-minute spatial resolution, the paper applies the spatial tobit regression model to estimate the possible impacts of improvements in transport accessibility in East Africa. It is found that rural accessibility and access to markets are important to increase agricultural production. In particular for export crops, such as coffee, tea, tobacco, and cotton, access to ports is crucial. The elasticities are estimated at 0.3–4.6. In addition, the estimation results show that spatial autocorrelation matters to the estimation results. While a random shock in a particular locality would likely affect its neighboring places, the spatial autoregressive term can be positive or negative, depending on how fragmented the current production areas are.