NGS Style Manual‎ > ‎- O -‎ > ‎

O, oh

Vocative O is always capped and never followed by a comma:
            O Lord, our strength
            Sail on, O Ship of State

The interjection oh, lowercased except at the beginning of a sentence, is preceded and followed by a comma unless it makes a unit interjection with another word:
  Oh, what a difference!
But, oh, what a difference!
Oh my, what a difference!
Oh yes, I agree.