In this study, we synthesized two series of novel 5-nitrofuran-2-carbohydrazides 21a–h and 22a–e in addition to a third series of thiophene-2-carbohydrazides 23a–g to develop potent antimicrobial and/or antitubercular agents. The newly synthesized compounds were evaluated in vitro for their antimicrobial and antimycobacterial activities. Most of the 5-nitrofuran-2-carbohydrazides 21a–h and 22a–e displayed variable activity against Aspergillus fumigates, Staphylococcus aureus, Streptococcus pneumonia, Bacillis subtilis, Salmonella typhimurium, Klebsiella pneumonia, Escherichia coli and Mycobacterium tuberculosis. The sulfonamide derivative 21f exhibited superior potency and broad-spectrum antimicrobial activity with minimum inhibitory concentration (MIC)=0.06–0.98 µg/mL and antimycobacterial activity with MIC=3.9 µg/mL. The 5-nitrofuran-2-carbohydrazides 21a, b, g, h and 22a–c exhibited significant antibacterial activity with MIC values in the range of 0.12–7.81 µg/mL. The significances of the 5-nitrofuran moiety and sulfonamide function were explored via the structure–activity relationship (SAR) study. In addition, docking studies revealed that the p-amino benzoic acid (PABA) and binding pockets of the dihydropteroate synthase (DHPS) were successfully occupied by compound 21f. Furthermore, two quantitative structure–activity relationship (QSAR) models were built to explore the structural requirements which controlled the activity.