In June of 1923, Captain E.J. King, Commander Submarine Division Three (later Fleet Admiral and Commander in Chief, U.S. Fleet, during WW II), suggested to the Secretary of the Navy (Bureau of Navigation) that a distinguishing device for qualified submariners be adopted. He submitted a pen-and-ink sketch of his own, showing a shield mounted on the beam ends of a submarine, with dolphins forward of, and aft of, the conning tower.

The suggestion was strongly endorsed by the Commander, Submarine Division Atlantic. Over the next several months, designs were solicited from several sources. Some combined a submarine with a shark motif. Others showed submarines and dolphins, and still others used a shield design. A Philadelphia firm, which had done work for the Navy in the field of Naval Academy class rings, was approached by the Bureau of Navigation with the request that it design a suitable badge. Two designs were submitted by the firm, and these were combined into a single design. It was a bow view of a submarine, proceeding on the surface, with bow planes rigged for diving, flanked by dolphins in a horizontal position with their heads resting on the upper edge of the bow planes.

Today a similar design is used: a dolphin fish flanking the bow and conning tower of a submarine. In March of 1924, the Chief of the Bureau of Navigation recommended to the Secretary of the Navy that the design be adopted. The recommendation was accepted by Theodore Roosevelt, Jr., Acting Secretary of the Navy in the same month.

The submarine insignia was to be worn at all times by officers and men qualified in submarine duty attached to submarine units or organizations, ashore and afloat, and not to be worn when not attached.

In 1941, the Uniform Regulations were modified to permit officers and men qualified who were eligible to continue to wear the submarine insignia after they had been assigned to other duties in the naval service, unless such right had been revoked. The officers' insignia was a bronze, gold plated metal pin, worn centered above the left breast pocket and above the ribbons and medals. Enlisted men wore the insignia, embroidered in silk, white silk for blue clothing and blue silk for white clothing. This was sewn on the outside of the right sleeve, midway between the wrist and elbow.

The device was two and three-quarters inches long. In 1943, the Uniform Regulations were modified to provide that "Enlisted men, who are qualified and subsequently promoted to commissioned or warrant ranks, may wear enlisted submarine insignia on the left breast until they qualify as submarine officers, at which time this insignia would be replaced by the officers' submarine pin."

In mid-1947, the embroidered device shifted from the sleeve of the enlisted men's jumper to above the left breast pocket. A change to the Uniform Regulations dated September 21, 1950 authorized the embroidered insignia for officers (in addition to the pin-on insignia) and a bronze, silver plated, pin-on insignia for enlisted men (in addition to the embroidered device).