Magic Mail
After the DEFCON conference earlier this year in Las Vegas, I watched David Copperfield’s show at the MGM Grand. I wanted to see him for years (he’s kind of a childhood hero of mine), but never managed to squeeze it into my schedule. This time, however, my return flight was a day later than usual so I used the opportunity to finally see the legend with my own eyes. What a wonderful, impressive show that was! The illusions were astonishing! I had a close-to-front-row seat, and since Mr. Copperfield was performing large parts of the show from within the audience space, I felt like being in the middle of the magic.
While there have been a few smaller illusions which I was able to find possible explanations for by thinking it through and Internet research, there were also tricks which to this day I can not get out of my head. How the … did he do that? I kid you not, I really believe to have seen a very large object appear in front of my eyes out of nowhere. I have a couple of theories, but every time I think those theories to the end something does not add up. There is always a piece of the puzzle missing. I might never understand. I know I can’t trust my eyes. But is any of this real magic? Let’s find out…
Technology’s Role In Illusions
Mr. Copperfield is famous for using cutting-edge technology during his shows and illusions. He has been seen examining the latest technology at the Consumer Electronics Show (CES). And he is believed to have used a computer-based approach for his famous flying illusion.
That was at a time when we were all still figuring out how to stabilize our half-a-pound quadcopters. And David Copperfield was moving his body through three-dimensional space hanging on an unknown number of super-thin strings. Strings that might have been pulled, balanced, and coordinated by a custom-made algorithm and computer. At least this is what people on the Internet believe.
Either way, technology plays an important role in Mr. Copperfields shows. And I pride myself on understanding a little bit of technology in general. I just had to find the one thing that I do understand well enough the regain my sanity. Luckily, I did not have to look far…
About Time (Travel)
At the beginning of the show, an assistant asked the audience to send an email from their mobile phones to Mr. Copperfield. We were then asked to store away our phones into a small, wooden box. Every table had one, just large enough to fit four to six mobile phones. A couple of minutes later the show started and Mr. Copperfield explained to us that he will be sending us an email in a moment. That email, he claimed, we cannot open before the end of the show because our phones are locked away in the wooden boxes. But we will be able to verify after the show that the email arrived at the beginning of the show and will contain information from the future. Time travel. Classic! I sat close enough to the stage to get a glimpse on the screen of his iPhone. I could see that he was not really using the email app but only pretending to do so. I decided to investigate the issue later and concentrate on the show first. Just in case I did not yet mention it: Awesome show! Worth every dollar! ✨🧙
At the end of the show, Mr. Copperfield unrolled a piece of paper that was placed above the audience at the beginning of the event. It was visible to us the whole time (but we did not pay attention to it the whole time, of course). He presented the unrolled paper to the audience, but the text was too small to read I think. He then turned and an assistant took a photo of him with the paper and the audience in the back. That photo was supposedly sent to us through time right into our inboxes on our mobile phones. We will have a closer look at this photo later.
When the show ended I pulled out my phone and immediately checked the email from the future. It was there! It even arrived in the past.
Note: When I took the screenshots I was in CET (GMT+1) while the show took place earlier this year when I was in PDT (GMT-7 DST). That may look like time travel too, but it is not. When I say from the future we are talking about the ~1.5h that the show ran. That is the time travel we have to investigate.
A Closer Look
For further examination, I switched to a real computer. 🧐 The Gmail app would not let me see the source code of the email. Here is the email in my regular Gmail inbox.
It even has an attachment, namely future.jpg
.
Even in a world of magic and the impossible, the future needs a file extension. There is just no such thing as an untyped future. 🔮 Who knew that JPEG is the future?
Let’s have a look at the message’s metadata:
The email arrived at the mail server shortly after the beginning of the show. It passed all checks for the sending domain and even had a valid DKIM signature. That means, it was neither messed with in transit nor was it forged.
Before we dig into the source code of the message we have to quickly refresh our knowledge on multipart messages and encoding.
- The Internet would be a boring place if we were not able to send rich text and multimedia messages. Imagine the many security issues we would not have run into if we had just stuck to plain text messages. How boring! Fortunately, clever engineers took on the challenge and invented the multipart MIME standard. That makes it possible to mix plain text and HTML messages. It also enabled attachments, which in turn gave rise to the practice of sending funny cat pictures around. Aren’t we lucky to have that? Multipart allows us to have multiple entities in a single message.
- The protocols that transport emails have been designed at a time when a byte was not necessarily the same width as an octet. That is, it was not safe to assume that a byte would always contain (at least) 8 bits. It was, however, safe to assume that email transporting systems would be able to deal with bytes of at least 7 bit. Modern humanoids (homo sapiens emojiens) not only need more bits per bytes but also demand more bytes per character to express their feelings. Clever engineers solved this by encoding 8-bit bytes into 7-bit bytes and calling this method base64. Base64 encoding allows us to embed binary attachments into a message.
Let’s run through a quick example to make sure we really understood how multipart and base64 work together to deliver an unprecedented, rich, multimedia web experience right inside the ancient email system. The source code of a message containing a greeting and a mysterious image attachment could look like this:
Mime-Version: 1.0
Content-Type: multipart/alternative; boundary="FOOBAR"
--FOOBAR
Content-Type: text/plain
Hi,
Nice to meet you.
Love, Dan
--FOOBAR
Content-Type: image/png; name="love.png"
Content-Transfer-Encoding: base64
iVBORw0KGgoAAAANSUhEUgAAAAkAAAAJCAYAAADgkQYQAAAAdklEQVQY02NggIKCgoL/BQUF/
9HZDAwMDIwwwQ52fgYGBgaGip8fGZDZEyZMYGSEKWBqrmb4V9vKgA4qfn5kYIJxsCmAASYGBg
aG9b++YJWEicPdZM7IzBDIxoOi4OT/vxA3IfsOphBZAdwkdIXICrAC5PCBAQAWM0Aepf7RRAA
AAABJRU5ErkJggg==
--FOOBAR--
Here, FOOBAR
is a multipart boundary. Each part of the message can have its
individual content type and encoding. To distinguish between the different parts we use
--FOOBAR
, which indicates the beginning of a new part. The last part is closed
by --FOOBAR--
. If we split the message up by --FOOBAR
we get a
text/plain
document and a base64-encoded image. Easy peasy, right?
The Message
Ok, now let’s have a look at the source code of Mr. Copperfield’s magic email from the future.
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="2f23b4c63f21481781e15eaf13e5f1ca"
--2f23b4c63f21481781e15eaf13e5f1ca
Mime-Version: 1.0
Content-Type: multipart/alternative; boundary="83fe1ab31eda4bb5ac763bdd6a872ad9"
--83fe1ab31eda4bb5ac763bdd6a872ad9
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="ascii"
Mime-Version: 1.0
photo.jpg
--83fe1ab31eda4bb5ac763bdd6a872ad9
Content-Transfer-Encoding: 7bit
Content-Type: text/html; charset="ascii"
Mime-Version: 1.0
[SOME_HTML]
--83fe1ab31eda4bb5ac763bdd6a872ad9--
--2f23b4c63f21481781e15eaf13e5f1ca
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="photo.jpg"
Content-Type: image/jpeg; name="photo.jpg"
Mime-Version: 1.0
--2f23b4c63f21481781e15eaf13e5f1ca--
That looks similar and different. Similar, because it’s just multiple parts. Different,
because we have nested multipart messages here. The outer parts’s boundary of
2f23b4c63f21481781e15eaf13e5f1ca
splits the message up into two parts:
- Part 1, which contains another (the nested) multipart message.
- Part 2, which contains a base64-encoded attachment.
We will look into part 1 shortly. For now, part 2 raises an interesting question.
The Attachment
Here is part 2 of the multipart message again:
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="photo.jpg"
Content-Type: image/jpeg; name="photo.jpg"
Mime-Version: 1.0
We expect a file photo.jpg
. We know that it is supposed to be base64-encoded.
But where is the actual data? This is a classic magic trick: We are fooled into believing
something is present while it is not. It is an illusion. There is no data! Our email client
is tricked into believing that this message has an attachment but it has not. It has just
the announcement of an attachment. Most email clients will fall for it and display
the attachment symbol.
While this trick may work on a email client it will not work on us. When we try to open the attachment we will see either an error message or nothing. It is, therefore, necessary to draw out attention elsewhere.
The Content
The rest of the message is another multipart message. That is, we have a
multipart/alternative
message within a multipart/mixed
message.
Let’s have a look:
Mime-Version: 1.0
Content-Type: multipart/alternative; boundary="83fe1ab31eda4bb5ac763bdd6a872ad9"
--83fe1ab31eda4bb5ac763bdd6a872ad9
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="ascii"
Mime-Version: 1.0
photo.jpg
--83fe1ab31eda4bb5ac763bdd6a872ad9
Content-Transfer-Encoding: 7bit
Content-Type: text/html; charset="ascii"
Mime-Version: 1.0
[SOME_HTML]
--83fe1ab31eda4bb5ac763bdd6a872ad9--
A multipart/alternative
message is a special type of multipart message. It
tells the reader (usually the email client) to present only one of the alternative
contents. This is usually used to send rich text (HTML) version of a message along with a
plain text version. Terminal email clients or those configured to not display HTML messages
would prefer the text part while fully featured email clients will go for the HTML part.
Either way, only one part in a multipart/alternative
message is supposed to be
displayed.
Here we have a plain text that just says photo.jpg
. I don’t know why Mr.
Copperfield included this string in the message. To me, it makes not much sense. We already
know the attachment is fake and this would draw attention to it instead of away from it.
In the previous source code sections, I omitted the actual text of the message and replaced
it with [SOME_HTML]
. It is time to lift the curtain and show you what really
was in the message (source edited for keeping secrets safe):
<div style="display: none; text-indent: -999px; color: #FFF;
overflow:hidden; width: 1px; height: 1px;">
[future.jpg]
</div>
HELLO,<br>
<br>
THANK YOU for coming to our SHOW.<br>
<br>
Remember, this letter, <span style="color: #d23f31;">MY WISH</span> was sent
at the BEGINNING of the show and LOCKED in your phone before any of these
RANDOM events took place...<br>
<br>
An IMPOSSIBLE EMAIL in your phone!<br>
<br>
<img style="margin-top: 20px;" width="100%"
src="http://s3-us-west-1.amazonaws.com/copperfield/shows2/2018/08/12/[random_name].jpg?[access_key]"
/>
There are two interesting tricks to see here:
- There is a reference to a file
future.jpg
which we did not see before. - The actual image is an URL. The tech-savvy reader will have guessed it long ago: The image from the future is not an attachment but is loaded from the web.
Let’s look at each of these tricks individually.
Hiding In Plain Sight
Remember how the email looked in the Gmail interface? It looked like we had an attachment
which is named future.jpg
.
But when we dug into the source code we did not see such a file. Only a file named
photo.jpg
.
The following code snippet contains the HTML DIV
element that makes all of
this posible:
<div style="display: none; text-indent: -999px; color: #FFF;
overflow:hidden; width: 1px; height: 1px;">
[future.jpg]
</div>
This is essentially a 1px
by 1px
DIV
that contains
white (#FFF
) text. It is not even displayed and the text is even moved out of
the invisible and small DIV
by applying a negative text-indent
of
-999px
.
The purpose of all this is to make the string [future.jpg]
appear in the
unparsed overview of an Inbox but disappear when the actual message content is parsed in
the reader view of an email client. The filename is hiding in plain sight! Another typical
illusion technique.
The Future’s Past
We learned that the actual picture from the future is loaded from the web. And by the time it is loaded, which is after the show, the file is not really from the future anymore but from the present. This trick is based on the lazy loading behavior of most email clients. It leverages the fact that inline pictures are not loaded until the user opens the message.
At this point I could have stopped as the secret of the trick was demystified. I could have. But I didn’t. Remember how I wrote that the paper sheet on the photo contains a summary of what had happened during the show? What if I can get my hands on the photo from another run of the same show? What if I run a pixel-wise diff on these two photos? What can I learn from that? What if I run a content-based diff on these two photos? Can I learn what parts of the show are truly random and which have a predefined outcome?
It was time to find out! On September 19th, according to the ticket webshop, another run of the show was scheduled in Las Vegas at 9:30 pm. That was September 20th at 6:30 am in Munich local time. I set an alarm clock to get up early and got ready to send one more email to Mr. Copperfield. I remembered the address I sent the first email to. And I roughly remembered how long after the start of the show the audience was encouraged to send the email out. All I had to do was to send my email, from my breakfast table in Munich, at the time the trick was scheduled in Las Vegas during the late evening show. I got lucky and shortly after I sent the message out I got a reply similar to the first one. But this time, my phone was not trapped in a wooden box. This time I was able to analyze the email while the show ran.
I opened the message and found a photo very similar to that of the first show. How similar was the photo? See for yourself:
On the paper sheet that Mr. Copperfield holds in his hands on the photo was a very generic description of the show. There was basically nothing randomized or non-deterministic. It was good enough to serve as a high-level summary of the show but would not trick anyone into thinking it was from the future.
While the show was running on the other side of the planet I enjoyed another cup of coffee. After ~2h I checked the photo URL again. This time the photo contained a more detailed description of the show. The generic parts were either replaced with show-specific data or removed entirely.
I was surprised by how few of the facts are actually specific to an individual show run. Much more than I expected is the magician in control over what happens during a show. Another interesting find was, that not every show is the same. There seem to be small variations and sometimes a trick is not being performed or another one is added. Interesting!
Possible Improvements
I am not an expert of magic shows, just a fan. Nevertheless, I think the trick can be improved in the following ways:
- The fake attachment is referenced as
future.jpg
in the hiddenDIV
but the file is namedphoto.jpg
in the multipart message. I am either missing something genius here or this has been a small laxness on Mr. Copperfield’s side. If so, it would be easy to fix. - The plain text version of the actual message contains the string
photo.jpg
and draws viewer attention to the broken attachment. I think something like Enable HTML to view this message! or Display Error would better hold up the illusion. Again, I might be missing something here. - For returning visitors to the show it is obvious that the picture from the future (without the text) looks the same every time. Only the text differs. It might be possible to have a set of pictures and randomize from that. This would decrease the odds for an individual to see the same picture twice. As this only applies to return visitors it is probably not worth investing much time into this.
- The reason I was able to fetch an earlier version of the
future.jpg
was, that I was able to retrieve the non-guessable link to the file by faking attendance. To prevent people from outside the audience to get the magic email Mr. Copperfield could do one of the following adjustments:- He could use a randomized, show-specific user part. Instead of always using
foo@example.com he could vary between
david@example.com
,show@example.com
,hello@example.com
, and so on. This would make it hard to guess the right user part of the email address for people who are not in the audience at the time of the announcement. - He could ask the audience to include a special keyword in the email. Any randomized or otherwise unpredictable word would work, e.g. the name of someone from the audience. Behind the scenes, an assistant would filter messages which do not include the correct keyword.
- He could use a randomized, show-specific user part. Instead of always using
foo@example.com he could vary between
- The photo could be stored on a custom server instead of an Amazon S3 bucket. Thus the server could respond with a status code of 503 (Service Unavailable) to requests from eager loading clients. This should trigger clients to reload the image if the user opens the email after the show, even if the phone tried to prefetch it in the meantime. More users would see the show-specific photo instead of the generic one.
Have A Great Time!
And now, after all the technobabble: Have magical winter holidays everyone and see you again next year! Consider visiting a magic show in the meantime…