The SARS-CoV-2 pandemic enables the analysis of immune responses induced against a novel coronavirus infecting immunologically naïve individuals. This provides an opportunity for analysis of immune responses and associations with age, sex and disease severity. Here we measured an array of solid-phase binding antibody and viral neutralising Ab (nAb) responses in participants (n=337) of the ISARIC4C cohort and characterised their correlation with peak disease severity during acute infection and early convalescence. Overall, the responses in a Double Antigen Binding Assay (DABA) for antibody to the receptor binding domain (anti-RBD) correlated well with IgM as well as IgG responses against viral spike, S1 and nucleocapsid protein (NP) antigens. DABA reactivity also correlated with nAb. As we and others reported previously, there is greater risk of severe disease and death in older men, whilst the sex ratio was found to be equal within each severity grouping in younger people. In older males with severe disease (mean age 68 years), peak antibody levels were found to be delayed by one to two weeks compared with women, and nAb responses were delayed further. Additionally, we demonstrated that solid-phase binding antibody responses reached higher levels in males as measured via DABA and IgM binding against Spike, NP and S1 antigens. In contrast, this was not observed for nAb responses. When measuring SARS-CoV-2 RNA transcripts (as a surrogate for viral shedding) in nasal swabs at recruitment, we saw no significant differences by sex or disease severity status. However, we have shown higher antibody levels associated with low nasal viral RNA indicating a role of antibody responses in controlling viral replication and shedding in the upper airway. In this study, we have shown discernible differences in the humoral immune responses between males and females and these differences associate with age as well as with resultant disease severity.