S.

could you be thinking of s. by j.j. abrams and doug dorst? it had a lot of margin notes because it was meant to be two people annotating the same book, and it utilized several different colours of "pen", as well as inserts, but based on the videos i saw, it may not have the upside down part that you're looking for. fwiw, this type of book is called "ergodic". you may be able to use that to help find the book you want! i was able to find this goodreads list with a lot of possible books.