Probably not the valve-stems then; that would be only when you first start it and they've let oil leak past whilst it's been sitting I think.
Oil rings on the piston is a possibility; you'll probably find the compression is fine as those rings are unaffected, but when the manifold is under vacuum (so definitely off-boost) that's when oil could get sucked past them and into the cylinder.
I wouldn't go stripping your engine down just yet though... I'm just taking a guess based on a conversation I had on the weekend.
Have you fiddled with the crank ventilation at all? Any chance it's just putting oil into the air intake instead of returning it to the sump?