Robusta coffee is the primary source of income for millions of smallholder farmers throughout the world’s tropics. The price smallholder farmers can get for their coffee is strongly influenced by bean characteristics (i.e. beans are of a sufficient size and have minimal defects). Climate is a key determinant of successful coffee production, but scant research has been undertaken to test and quantify climate impacts on robusta coffee bean physical characteristics. Here we investigate how climate relates to the risk of poor coffee bean characteristics in one of South East Asia’s key coffee producing areas, the central highlands of Vietnam. We use 5 years (2012–2016) of coffee bean characteristic data from 60 farms. Hierarchical modelling was used to investigate how rainfall and temperature related to two indicators of coffee bean characteristics (1) the probability of below average coffee bean size and (2) the probability of above average coffee bean defects. Low rainfall (<1600 mm) during the late growing season (July-September) greatly increased the risk (>80% probability) of below average coffee bean size. Conversely, high rainfall (>750 mm) and high mean minimum temperature (>22 °C) during harvest (October-December) increased the risk (>75% probability) of above average coffee bean defects. Various coffee bean characteristic subcomponents (e.g. insect damage and mouldy beans) and different bean sizes were also examined and were affected by a range of rainfall and temperature predictors across the flowering, growing and harvest seasons. With this information targeted risk-management strategies (e.g. targeted irrigation during hot and dry growing seasons, adjusting harvest timing and employing drying techniques during wet and cold harvest periods) could be developed to minimise the effect of climate conditions that increase the risk of coffee bean defects. Successfully managing the impacts identified here, could decrease coffee bean defects and in turn increase the incomes of smallholder coffee farmers.