I discovered the source of the problem. My camera is storing camera orientation information in the EXIF data of the jpeg files. Right clicking a jpeg in Windows Vista doesn't show the orientation field, which is why I didn't discover this earlier. Picasa is rotating the image when uploaded based on the stored orientation. This doesn't seem like the correct behavior, especially because the behavior is not consistent across browsers. I am hoping Google fixes this, or at least provides a way to change the rotation once uploaded.
I have been involved in a discussion about this issue in the Blogger help groups. A Blogger employee just followed up on that discussion hinting that they are hopefully going resolve this issue!
Thanks a lot for investigating; it looks like you have stumbled into a solution for some of the photo issues that have arisen in the past here. I'll pass your feedback along to the rest of the team and we'll see if we can make this better.
And thanks to Mishka for the heads up!
The Blogger Team