Materials improvement is a powerful approach to reducing loss and decoherence in superconducting qubits, because such improvements can be readily translated to large-scale processors. Recent work improved transmon coherence by using tantalum as a base layer and sapphire as a substrate1. The losses in these devices are dominated by two-level systems with comparable contributions from both the surface and bulk dielectrics2, indicating that both must be tackled to achieve substantial improvements in the state of the art. Here we show that replacing the substrate with high-resistivity silicon markedly decreases the bulk substrate loss, enabling 2D transmons with time-averaged quality factors (Qavg) of 9.7 × 106 across 45 qubits. For our best qubit, we achieve a Qavg of 1.5 × 107, reaching a maximum Q of 2.5 × 107, corresponding to a lifetime (T 1 ) up to 1.68 ms. This low loss also allows us to observe decoherence effects related to the Josephson junction, and we use an improved, low-contamination junction deposition to achieve Hahn echo coherence times (T 2E ) exceeding T 1 . We achieve these materials improvements without any modifications to the qubit architecture, allowing us to readily incorporate standard quantum control gates. We demonstrate single-qubit gates with 99.994% fidelity. The tantalum-on-silicon platform comprises a simple material stack that can potentially be fabricated at the wafer scale and therefore can be readily translated to large-scale quantum processors.