"a quick blow, dig, or thrust with the fist," by 1570s, probably from punch (v.). In early use it also could refer to blows with the foot or jabs with a staff or club. Originally especially of blows that sink in to some degree ("... whom he unmercifully bruises and batters from head to foot: here a slap in the chaps, there a black eye, now a punch in the stomach, and then a kick on the breech," Monthly Review, 1763).
The figurative sense of "forceful, vigorous quality" is recorded from 1911. Punch line (also punch-line) is from 1915, originally in popular-song writing. To beat (someone) to the punch in the figurative sense is from 1915, a metaphor from boxing (where it is attested by 1913). Punch-drunk "dazed from continued punching, having taken so many punches one can no longer feel it" is from 1915 (alternative form slug-nutty is from 1933; compare sleep-drunk, 1889, "confused and excited while being half awakened from a sound sleep").