After some surfing around the internet, I found an Amazon page with explanations about Kindle Cloud Reader and Firefox. In Firefox you need to manually adjust the permissions for the Kindle Cloud Reader in order to store books offline.
Instead of copying and pasting the instructions, which then quickly will be outdated, I'll just paste the link to the Amazon page here: Amazon Help: Enable offline reading. The section on Firefox is clear and easy to follow.
It didn't, however, enable me offline access to my books (meaning: without internet connection, I can't open my book although it's supposed to be available offline).
I found this forum post (which deteriorates in petty fights but still has some interesting info) which mentioned the necessity of turning private browing on - doesn't make a difference in my case -; and this post that mentions installing a Kindle app - which doesn't exist for Linux so no solution either.
So, as far as I'm concerned, at this point in time (May 2014) Kindle Cloud Reader doesn't work offline with Firefox on Ubuntu. If you've got it working with Firefox on Linux (tested and tried!), by all means share. Chrome works out of the box.