View Discussion Show
Improve Article Save Article View Discussion Improve Article Save Article POP3 (Post Office Protocol 3) and
IMAP (Internet Message Access Protocol) both are MAA (Message accessing agent), both of these protocols are used to retrieve messages from the mail server to the receivers system. Both of these protocols are accounted for spam and virus filters. IMAP is more flexible and complex than POP3. Difference Between POP3 and IMAP :
In this post, we’re going to dive into IMAP. IMAP stands for Internet Message Access Protocol, and it’s the open standard that describes how to access messages in an email mailbox. While IMAP is an important part of receiving emails, it’s not always the easiest thing to implement (or understand), which is why Nylas exists! In short, IMAP is the protocol that email clients like Mail.app, Thunderbird, and Mailspring use to download messages from your email account and to make changes like archiving messages or sorting them into folders. A Brief History of IMAPIMAP was originally created by Mark Crispin at Stanford in the 1980s. Crispin later moved to the University of Washington and spent 20 years there working on the IMAP specs and reference implementation. The IMAP specification comes in the form of a “Request for Comments”, or RFC, which is basically a memo describing how to implement the protocol that has been adopted as a standard by the Internet Engineering Task Force. RFCs may be revised to make clarifications or changes, and the current RFC that describes IMAP is RFC3501 which was published in 2003. That’s right: the protocol most mail clients use to sync email is over 15 years old! IMAP was initially designed as a better alternative to an older protocol called POP, with the goal being “to permit manipulation of remote mailboxes as if they were local”—specifically, POP’s mode of operation was to download mail to your local machine and delete it from the server, which made it difficult to manage your mail if you were using multiple computers. This was no problem in the days of timesharing mainframes, but became one as personal computers became more prevalent and you might have a PC at home as well as at work, and wanted to browse your email from either place. The original version of IMAP is lost to history, and IMAP2 (released in 1988) is the first version that made it into the standards. This first version of IMAP only supported “online” operation: it assumed that your mail client would be connected to the server when viewing or modifying messages. You couldn’t use it to sync a copy of your mailbox locally and then update your local copy when you reconnected, because messages were only identified by a “sequence” number, and sequence numbers were not persistent across client sessions. One can only imagine that in those days folks figured that it’d always be the case that whenever you wanted to check your email, you’d be sure to have a reliable network connection. My guess is that it was because computing from mainframes and timesharing systems was essentially logging directly in to a server machine, and meant you had an always-on Internet connection at your disposal. Fast forward a few years to the dialup era, and it became apparent that hogging your phone line constantly so you could read your email just wasn’t going to cut it. Enter support for “disconnected” operations, which was bolted on to IMAP with the introduction of IMAP4 in 1994. This revision to the protocol added persistent message identifiers (called “UIDs”) and an entire draft spec explaining how you could sync changes made to locally cached data to data stored on a server using IMAP commands. We’ll talk more about this later, but this evolution of the protocol over time is a major reason that implementing IMAP clients is complex in the modern era, as disconnected operation is a required feature for modern apps. The basic IMAP protocol has remained unchanged since 1996, with all new features since then being implemented in optional extensions to the protocol, some of which have become standards—but adoption of even the most common extensions still varies wildly across email service providers. How IMAP WorksNow that you know some of the history of the protocol, we’re going to interactively introduce you to the nuts and bolts of how it works. We’re going to connect to an email account, list all of the available folders in the mailbox, and download a message. Along the way, we’ll identify and explain the relevant parts of the protocol needed to understand what is happening. You’ll be able to follow along as we walk through the protocol, as long as you have access to a terminal, the TCP, HTTP & IMAPMany modern client-server protocols take place between a web browser and a server, or an app and an API. These APIs are all implemented on top of a base protocol, HTTP (“hypertext transfer protocol”), which defines the semantics of different types of requests—GET, PUT, POST etc. HTTP is in turn implemented on top of a lower-level protocol called TCP (“transmission control protocol”), which simply guarantees that packets of information are delivered reliably to the destination. IMAP, like HTTP, is implemented on top of TCP—and it defines the semantics of different types of requests, called IMAP commands. IMAP in general is a lot more concise than HTTP, which was key for IMAP in the early 90s as network connections were very low-bandwidth. IMAP Step by StepLet’s get started! Follow along at home in your terminal by typing the commands, subbing in the details of your email account where relevant. Note that if you’re using Gmail, you may need to disable protection against logins from “less secure apps” for this to work. The first thing any client needs to do is to to make a connection to the remote server on a specific port. These ports can vary because mail servers first accepted plaintext connections and later added support for secure encrypted connections. You can generally find the settings for your provider by checking their help articles. For example, here’s where to find the settings for Gmail. (If you’re curious as to the background of why different ports are used, we recommend this fantastic guide from Fastmail.) Got your settings to make a secure connection? Let’s go! On the command line, we can connect using the $ openssl s_client -connect imap.gmail.com:993 -crlf [... certificate verification removed for brevity ...] * OK Gimap ready for requests from 216.38.147.18 h2mb556258870pjq In the above command, CommandsIMAP commands generally look like this: <tag> <command> [<arg1><arg2>…] We’ll explain these parts in the context of an example command, called CAPABILITY. CAPABILITY allows you to ask a server which IMAP extensions it supports: tag1 CAPABILITY * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 XYZZY SASL-IR AUTH=XOAUTH2 AUTH=PLAIN AUTH=PLAIN-CLIENTTOKEN AUTH=OAUTHBEARER AUTH=XOAUTH tag1 OK Thats all she wrote! h2mb556258870pjq Wahoo, we just communicated with an IMAP server! Let’s explain what we sent and what we got back. A command is similar to a request in HTTP. It’s telling the server to do something or asking it for information. In this case, we’re asking the server to tell us which capabilities it supports. More on capabilities in just a bit. The CAPABILITY command has no arguments, but if it did, we’d simply type them in after the command name and before we press enter. TaggingTags ( This tagging ability means that it’s possible for a server to handle more than one request at the same time from a client on the same connection, and to indicate their completion by sending back the appropriate tag. In practice, many clients don’t support this ability to send concurrent requests, and simply block waiting for data to arrive on the open socket after a command is sent. CapabilitiesSo what does this list of “capabilities” mean, anyway? A
“capability” is a short name for a feature the server supports. Some of these capabilities, like AuthenticationHere’s how you authenticate your mail account to an IMAP server: tag2 LOGIN [email protected] tag2 NO [AUTHENTICATIONFAILED] Invalid credentials (Failure) Server ResponsesAs you can see, server responses don’t always say
The server is free to add a bit more information after the response if it so desires,
like tag3 LOGIN [email protected] * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH UTF8=ACCEPT LIST-EXTENDED LIST-STATUS LITERAL- SPECIAL-USE APPENDLIMIT=35651584 tag3 OK [email protected] authenticated (Success) As you can see, after a successful login, the server sends the client the list of capabilities again, showing what new capabilities have been unlocked as a result of the login and which old ones have gone away. Some standard extensions in this list are Listing FoldersOK, now that we’re logged in, let’s try to do something useful. First, let’s check which folders exist in the mailbox: tag4 LIST tag4 BAD Could not parse command Whoops! As you can see, if you get the arguments wrong to a command, the server will reject it and say tag5 LIST "" "*" * LIST (HasNoChildren) "/" "Call log" * LIST (HasNoChildren) "/" "INBOX" * LIST (HasChildren Noselect) "/" "[Gmail]" * LIST (All HasNoChildren) "/" "[Gmail]/All Mail" * LIST (Drafts HasNoChildren) "/" "[Gmail]/Drafts" * LIST (HasNoChildren Important) "/" "[Gmail]/Important" * LIST (HasNoChildren Sent) "/" "[Gmail]/Sent Mail" * LIST (HasNoChildren Junk) "/" "[Gmail]/Spam" * LIST (Flagged HasNoChildren) "/" "[Gmail]/Starred" * LIST (HasNoChildren Trash) "/" "[Gmail]/Trash" * LIST (HasNoChildren) "/" "alabel" tag5 OK Success Folder FlagsHere we can see the folder hierarchy that exists in the mailbox. Folders have special flags, indicated in parentheses. Some of these flags are useful for traversing the folder hierarchy, like SessionsOK, let’s read some email now. First, because IMAP connections are stateful, we need to open a session on the folder that we care about using the tag6 SELECT "INBOX" * FLAGS (Answered Flagged Draft Deleted Seen $NotPhishing $Phishing) * OK [PERMANENTFLAGS (Answered Flagged Draft Deleted Seen $NotPhishing $Phishing *)] Flags permitted. * OK [UIDVALIDITY 1] UIDs valid. * 55 EXISTS * 0 RECENT * OK [UIDNEXT 54948] Predicted next UID. * OK [HIGHESTMODSEQ 11865059] tag6 OK [READ-WRITE] INBOX selected. (Success) There’s a lot of information the server
sends us here! Sessions are a key component of how IMAP works, and a session lasts from when you When opening a session, this server has sent us, in addition to the OK indicating success:
A few of these items are going to require a bit more explanation. First, UIDs. UIDs & Sequence NumbersA reminder: a long long time ago, there was IMAP2, and in IMAP2 messages were identified by sequence numbers. Sequence numbers were literally just that: monotonically increasing numbers, one for each message in a mailbox: tag7 SEARCH ALL * SEARCH 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 tag7 OK SEARCH completed (Success) Every time you selected a mailbox, if there were new messages, there would be more sequence numbers. And if you moved messages around and started a new session, the sequence numbers assigned to messages might be different from the ones from the old session. This very simple scheme worked pretty well if all you did was connect, read your mail online, and disconnect. But it didn’t work if you wanted to connect, download all of your mail locally for offline access, and then reconnect later and have your client sync its local state with the state on the server—because these non-persistent sequence numbers didn’t allow clients to map local mailbox state to remote mailbox state, because messages on each side might have different IDs! Enter IMAP4 and UIDs. UIDs (unique IDs) are persistent to a message as long as it stays in the same folder and the server’s UIDVALIDITY integer has not increased. So when a client starts a new session on a mailbox, it has to check the UIDVALIDITY and compare it to its local cached value, and if the server’s UIDVALIDITY is higher, it must throw away all cached local UIDs and resync from scratch. But otherwise, it’s free to compare its local cached data to the data on the server and only sync changes. To retain backwards compatibility, IMAP4 supports both
sequence numbers and UIDs, and you can ask for UIDs by prefixing any command that returns message identifiers with tag8 UID SEARCH ALL * SEARCH 53809 54180 54185 54211 54222 54268 54410 54451 54452 54453 54463 54527 54557 54595 54596 54602 54608 54617 54637 54639 54643 54664 54667 54670 54673 54699 54707 54738 54753 54754 54804 54806 54847 54852 54857 54864 54873 54877 54880 54881 54883 54888 54895 54899 54902 54903 54904 54905 54906 54916 54923 54926 54931 54942 54947 tag8 OK SEARCH completed (Success) As you can see, unlike sequence numbers, UIDs are not guaranteed to be increasing in increments of one—gaps develop in the UIDs assigned to a mailbox as messages arrive and are moved around between folders. StatefulnessWhile we’re here, one aside about statefulness: Generally, statefulness is not a desirable feature in protocols. Statefulness
makes it trickier to use connection pools with IMAP, because the Fetching Message DataOK let’s finally actually download data from a message! tag13 UID FETCH 53809 (FLAGS BODY[HEADER.FIELDS (DATE FROM)]) * 1 FETCH (UID 53809 FLAGS (Answered Seen) BODY[HEADER.FIELDS (DATE FROM)] {80} From: My Friend <[email protected]> Date: Sun, 18 Feb 2018 10:50:34 -0500 ) * 55 EXPUNGE * 54 EXISTS tag13 OK Success IMAP provides lots of knobs for allowing clients to request partial data about messages, including the size of a message, flags (like There are just a couple more concepts we want to cover about IMAP before we wrap up. Unsolicited responses & IDLEYou may have noticed some strange untagged responses in the previous example:
This is actually a notification about one message disappearing from the mailbox and another message appearing while we’ve had the mailbox open. It’s valid in IMAP for the server to send these unsolicited messages to a connected client at any time. There’s also a mode you can put an IMAP connection into, called IDLE, which is essentially only for handling notifications about changes to a mailbox, and is often used by clients to quickly sync new messages coming in. Disconnected syncingDisconnected syncing allows email apps to work well in the face of airplane wifi and bad cellular connections. It’s not so much a first-class feature of IMAP as a thing that you can build on top of the protocol, with significant effort, because IMAP wasn’t originally designed to support it. The way to do a disconnected sync with IMAP4 in its original form was pretty simplistic: you’d have to take the list of UIDs you have locally for each folder and compare it to the list of UIDs currently in the folder on the server, and fetch new messages and delete old ones based on the differences. As mailboxes grew, this became more and more complex for clients to handle, and later extensions like CONDSTORE and QRESYNC improved the flow for disconnected resyncs. But even with these extensions, the process for doing a disconnected resync is complicated to implement—and even today many IMAP server implementations don’t support the newer extensions, so general-purpose clients must contain conditional logic for different servers. The general reference for how to implement disconnected syncing on top of IMAP is RFC4549. SummaryAlright, we’ve done it! Through following along in this post, you’ve learned about the history and major components of IMAP, an open protocol for accessing and manipulating remote mailboxes. We’ve connected to an IMAP server, logged in to our email account, listed folders on the remote server, and downloaded message data. This is exactly what happens under the hood in email apps as you browse your mailbox. We also learned that because of how IMAP developed over the years, disconnected syncing is fairly complicated for clients to implement—which is why we built our delta API to simplify the process. Next time, we’ll be diving into the other vital open protocol for email: SMTP, which is how emails are sent. We hope you’ll stick around and join in on the fun! Which of the following port numbers are reserved for NetBIOS services select 3 answers?Cards In This Set. What physical security controls provide protection against laptop theft?Portable devices such as laptops are particularly vulnerable to theft, loss, and resale, and should be properly secured with a lock. Most laptops and desktop computers have built-in slots made to connect with a cable lock. These locks are available at most computer stores.
Which of the following describes the function of a digitizer?A digitizer is a hardware device that receives analog information, such as sound or light, and records it digitally. Usually, the information is stored in a file on a computing device. This process is called digitization.
Which type of information can typically be synchronized between devices running Windows 10?Synchronized data might include contacts, applications, emails, pictures, music, video, calendars, bookmarks, documents, location data, social media data, e-books, or passwords. Examples of synchronization methods include synchronizing mobile devices to the cloud, desktop, or automobile console.
|