I ran across an interview of Mark Twain by Rudyard Kipling, that contains this solid advice from Twain, on conscience:

Your conscience is a nuisance. A conscience is like a child. If you pet it and play with it and let it have everything that it wants, it becomes spoiled and intrudes on all your amusements and most of your griefs. Treat your conscience as you would treat anything else. When it is rebellious, spank it—be severe with it, argue with it, prevent it from coming to play with you at all hours, and you will secure a good conscience; that is to say, a properly trained one. A spoiled one simply destroys all the pleasure in life. I think I have reduced mine to order. At least, I haven't heard from it for some time.