It Takes Two to Tumble by Cat Sebastian

Some of Ben Sedgwick's favorite things:

Helping his poor parishioners
Baby animals
Shamelessly flirting with the handsome Captain Phillip Dacre

After an unconventional upbringing, Ben is perfectly content with the quiet, predictable life of a country vicar, free of strife or turmoil. When he's asked to look after an absent naval captain's three wild children, he reluctantly agrees, but instantly falls for the hellions. And when their stern but gloriously handsome father arrives, Ben is tempted in ways that make him doubt everything.

Some of Phillip Dacre's favorite things:

His ship
People doing precisely as they're told
Touching the irresistible vicar at every opportunity

Phillip can't wait to leave England's shores and be back on his ship, away from the grief that haunts him. But his children have driven off a succession of governesses and tutors and he must set things right. The unexpected presence of the cheerful, adorable vicar sets his world on its head and now he can't seem to live without Ben's winning smiles or devastating kisses.

In the midst of runaway children, a plot to blackmail Ben's family, and torturous nights of pleasure, Ben and Phillip must decide if a safe life is worth losing the one thing that makes them come alive.

This book is part of the "Seducing the Sedgwicks" series.
Here are some other books from this series: