Ok I figured it out. iPhones and iPads don't actually rotate the pictures so up is up. Thy record a bit of data to the file that says which side is up. But windows does not read that data, so I have to take all pictures with the volume buttons facing down face palm, really Apple.