The effect of the Japanese diet on cancer incidence remains unclear. The purpose of this study was to examine the association between the Japanese diet and the risk of all-cause and site-specific cancer. We analysed 14-year follow-up data from the Osaki Cohort study of 25,570 Japanese men and women aged 40–79 years. The Japanese diet was evaluated using a 39-item food frequency questionnaire at baseline. Based on a previous study, we used eight food items to calculate the Japanese Diet Index (JDI) score: rice, miso soup, seaweed, pickles, green and yellow vegetables, seafood, green tea, and beef and pork. The participants were divided into quartiles based on their JDI scores. The Cox proportional hazards model was used to estimate the hazard ratios (HRs) and 95% confidence intervals (CIs) of cancer incidence. During the mean 10.4 years of follow-up, we identified 3,161 incident cases of all-cause cancer. Multivariable analysis showed that the JDI score was not associated with cancer incidence. In comparison to Q1 (the lowest), the multivariable HRs and 95%CIs were 1.01 (0.92–1.12) for Q2, 0.94 (0.85–1.04) for Q3, and 1.06 (0.95-1.18) for Q4 (the highest). Furthermore, separate analyses of nine common cancer sites demonstrated no association with the JDI score. The results were consistent even after a sensitivity analysis using multiple imputation. This prospective study showed that the Japanese diet was not associated with cancer incidence. The results suggests that the Japanese diet could contribute to a person’s overall health and well-being without increasing cancer risk.