We study analytically the quantum thermalization of two coupled two-level systems (TLSs), which are connected with either two independent heat baths (IHBs) or a common heat bath (CHB). We understand the quantum thermalization in eigenstate and bare-state representations when the coupling between the two TLSs is stronger and weaker than the TLS-bath couplings, respectively. In the IHB case, we find that when the two IHBs have the same temperatures, the two coupled TLSs in eigenstate representation can be thermalized with the same temperature as those of the IHBs. However, in the case of two IHBs at different temperatures, just when the energy detuning between the two TLSs satisfies a special condition, the two coupled TLSs in eigenstate representation can be thermalized with an immediate temperature between those of the two IHBs. In bare-state representation, we find a counterintuitive phenomenon that, under some conditions, the temperature of the TLS connected with the high-temperature bath is lower than that of the other TLS, which is connected with the low-temperature bath. In the CHB case, the coupled TLSs in eigenstate representation can be thermalized with the same temperature as that of the CHB in nonresonant cases. In bare-state representation, the TLS with a larger energy separation can be thermalized to a thermal equilibrium with a lower temperature. In the resonant case, we find a phenomenon of anti-thermalization. We also study the steady-state entanglement between the two TLSs in both the IHB and CHB cases. ; Comment: 14 pages, 5 figures