DiPot: Ice Tea tech » server

Webmail vs IMAP4 vs POP3. Or not?

ageor — Tue, 16 Nov 2010 22:14:24 +0000

With email dominating a good part of our communication, mailbox management is an ever growing issue. This is primarily intended for people who access their email from at least two different email clients (webmail counts as one) and people who share a mailbox with others.

I want access to my email via webmail and offline acceess on my two laptops and mobile phone. Worse than that, I am using 2 different email clients on each laptop (one I regularly use and one I am trying). So, I frequently get to answer the question “How to best setup my client(s) to receive email; POP3 or IMAP4?” for myself and others. As usual, there is no single answer. However, there may be a proper email setup strategy for some scenarios. Good understanding of the tools will, hopefully, help determine which one it is. First, some background:

Webmail

Webmail is actually a web-based email client. As such, it is accessible from a web browser and it talks directly to the email server instead of transferring emails locally.

Webmail can be a one stop shop, provided there is access to the email server, which probably also requires Internet access. Yahoo! mail, Hotmail, Gmail, the upcoming Facebook Social Inbox etc. are full fledged email clients that can receive, send and manage emails, complete with antispam, folders and address books. A webmail user has no need to know about protocols to enjoy complete use of an email system, unless he would also like to access email offline. In that case, use of an email client like Pegasus, Eudora, Outlook Express, Outlook (yes, they are different), Evolution, Thunderbird etc. (forgive me if I missed your mailer of choice) is required.

SMTP (Simple Mail Transport Protocol)

Not to be forgotten, SMTP is the protocol that sends email. It is used by all email clients in both POP3 and IMAP4 configurations.

POP3 (Post Office Protocol v3)

POP3 ia a protocol for one way transfer (reception, downloading) of email from the server to the client for offline use. It is supported by all of the above (and more) mailers.

Once an email is succesfully received by the email client, it is, typically, erased from the mailbox at the server. Thankfully, most mailers allow the user to modify this behaviour so the mailer leaves the email at the server’s mailbox as well, either indefinitely or for a preset amount of days. Not erasing an email from the server is so important that Gmail (basically a webmail system) supports setting emails to not be erased on the server side, in case email clients are setup wrong.

So, when we need to have our mail delivered to more than one clients, we can set them up so they don’t erase downloaded emails from the server. This way, email stays on the server ready to be downloaded again or to be accessed via a webmail client.

IMAP4 (Internet Message Access Protocol v4)

IMAP is also supported by all of the above (and more) mailers. It is usually perceived as another protocol to receive email. Well, it is not! It ia actually a protocol to synchronize folders on the email server with folders on the email client(s). Email reception is, effectively, a subset of that.

So, what exacly is the difference and why is it important? Here is a partial list (visit a more comprehensive list and “Internet Message Access Protocol” in Wikipedia). The reason I use “differences” over “advantages” is that advantages depend on the usage scenario:

	Some IMAP and POP protocols differences		Best protocol for scenario
	IMAP	POP	User with multiple email clients	Multiple users sharing a mailbox
Mailbox Size	The complete mailbox resides on the server and requires the corresponding disk space. Depending on how email is synchronized in the clients, they may require less disk space than on the server	At worst, only the size of the “Inbox” folder, provided that mail is not deleted when received (default behaviour)	POP	POP
Folders Structure	The structure of the email folders is the same in the server and the clients and any change propagates from the server to all the clients. If you like to arrange your emails in folders, you only have to do it once for all your clients and the server. Only folders subscribed-to are synchronized	The structurs must be maintained on every client separately	IMAP	IMAP
Direction	Emails get copied both ways; from the server to the client(s) and from the client(s) to the server. This applies to all folders (including “Sent Items” and “Drafts”). It also applies to any number of clients	Email transfer goes only from the “Inbox” folder on the server to the client(s)	IMAP	IMAP
Email Status Synchronization	Email status of “Deleted”, “Read” and “Starred” synchronizes as well. When it changes, it first propagates to the server and from there to all clients	Some clients can direct the server to delete emails that are deleted locally. “Read” and “Starred” status is local to the clients and to the server and does not synchronize in any direction	IMAP	POP
Deletion	Email deletions move both ways. Once a deletion happens on or reaches the server, it propagates to all the clients	Clients never ask for the same email again. Once downloaded, nothing will happen to it in the client, even if deleted on the server	IMAP	POP
Push email delivery	Yes, when both server and client support the IDLE command	No	IMAP	IMAP

User with multiple email clients

IMAP is clearly better for a single user with multiple clients, provided there is enough room for the mailbox on the server. Even with a limited mailbox, a backup of older emails will allow deletion from the server. This backup can be available in as many of the clients as needed.

Multiple users sharing a mailbox

Complete synchronization, a clear advantage for a single user, turns into a disadvantage when multiple users share an email account. I wouldn’t want my collegue(s) to erase the emails he doesn’t like for me as well. I have my own important emails to star. I don’t want to have emails I haven’t read yet appear to be read because someone else read them. And I have my own way to organize my inbox. I want my own email statuses and organization. But I also need to have access to the emails he composes and sends through this account. This is the one synchonization I need.

Thankfully, a mixed IMAP and POP system can be configured to optimize for all needs. The solution I suggest to this (quite frequent, I may add) problem is: Set it up twice. Once as POP3 without deletion for email reception and once as IMAP for the “Sent Items” (or “Sent”) and “Drafts” folders synchronization. Care is needed to not send email from the POP edition of the account. Even then, if the sent email is manually or automatically -via a rule- moved to the IMAP “Sent” folder, the result will be the same. Please note that while sent items and drafts will synchrnize back to the server, any folder-based organization of the email won’t happen on the server, therefore it won’t appear during mailbox access via webmail.

Multiple users sharing a mailbox and using multiple email clients

Unfortunately, the second scenario doesn’t mix well with the first. If I want to organize my shared mailbox in folders and access it from multiple clients, I will have to do so manually on each client. This is a scenario where incoming mail needs to arrive to a mailbox all my own. Actually, nothing a good old mail forwarding setup can’t do, at the expense of multiple times the incoming email space on the server…

How many devices and email clients do you use? Are you happy with your setup? Please share!

Possibly related posts (auto generated)

WordPress: External RSS Feeds Optimization with KB Advanced RSS

ageor — Wed, 22 Apr 2009 19:00:10 +0000

Apart from its own Ice Tea tech blog, Dipot.com also displays two external feeds relating to Optimization and Standards. I needed to optimize the post titles of these two feeds by getting rid of their long suffixes, both for better looks and for more content within the same space.

The feeds were served by the standard RSS WordPress 2.7.1 widget, which is fine, except it does not support much customization. After some search, I found Advanced RSS 1.0.1 WordPress plugin and KB Advanced RSS 2.1.3 WordPress plugin. Smart RSS only works until WP 2.7.0 and I didn’t want to tweak with that. Advanced RSS only works until WP 2.6.2 and requires PHP5, DOM and XSL extensions loaded on the server and I don’t want to depend on those if I don’t need to, so the choice was easy.

Installation of KB Advanced RSS worked flawlessly right from within wp-admin. Then, on my first attempt to configure it, I came across a problem which could very well had been a show stopper: for some reason, except for the feed’s URL and Title, the widget would not commit any of the other parameters to the “Options” table of the database. Troubleshooting this issue, I ended up reading all the users’ comments on the plugin’s site. As usual, I was not alone; this is a common issue. Thankfully, Wesley of http://www.improvingtheweb.com had already posted the solution to PHP4 not committing widget field changes in WordPress. The solution worked like a charm. And, as expected, when I later upgraded to PHP5 the problem wasn’t even there to begin with.

KB Advanced RSS is well thought-out, very straight forward with good documentation and examples. It supports all the features of the standard widget (including multiple instances) plus formatting and full support of all the feed’s fields to display or not (including the debugging tool to show which these fields are)! It further supports two modifiers to optionally manipulate the string before display, by taking out a number of characters from the start (ltrim) and/or trimming it to a specific length (trim).

Alas, what I actually needed was to take out a number of characters from the end of the string. Fortunately, as it turned out, KB Advanced RSS’s code is pretty straight-forward and extending it with my own modifier (rtrim) to do just that was only a matter of 3 lines of PHP code added to the version of the plugin that runs by my version of Wordpress; there’s one for versions up to 2.7 (pre-wp-2-7.php) and one for versions after that (post-wp-2-7.php). Here is the PHP code from the modified post-wp-2-7.php (my rtrim extension is in blue):

///////////////////////////////// CUSTOMIZATIONS /////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////
// If you want to write customizations, put them here. The variable to modify is $text. See the FAQ.
/////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////

// right trimming
// cuts $rtrim characters from the right
// by ageor, April 9, 2009
$rtrim = (is_numeric($rtrim) && 0<$rtrim) ? (int) $rtrim : null;
if (is_int($rtrim))
$text = substr( $text, 0, strlen($string)-$rtrim );

// left trimming:
$ltrim = (is_numeric($ltrim) && 0<$ltrim) ? (int) $ltrim : null;
if (is_int($ltrim))
$text = substr( $text, $ltrim );

// length trimming (do after left trimming)
$trim = (is_numeric($trim) && 0<$trim) ? (int) $trim : null;
if (is_int($trim))
$text = substr( $text, 0, $trim );

return $text;

The way this is written, the string in question can undergo any mix of all three trimming types necessary.

All setting up and troubleshooting took place by adding one instance of the widget and configuring it, then a second instance. When all was done and I had what I wanted, I removed the original two instances of the original RSS widget, positioned the new ones on top and saved, certain I was done. However, as the resulting “Widgets” Wordpress admin page suggested after saving, something was very wrorg. It displayed the message ‘You are using 3 widgets in the “Left Sidebar” sidebar’ but not the widgets themselves, nor the “Save Changes” button. Sure enough, I ended up with an unworkable site, as these where the first 2 widgets on the left column of Ice Tea tech and the page that appeared ended with the title banner. Disaster!

To cut a long story short, plenty of “trial and error”s later and with the help of Vicki of neTWire, here is my conclusion: For KB Advanced RSS widget to work with WP 2.7.1 and the Atahualpa WordPress theme, the WordPress RSS widget needs to be activated and positioned above it. That’s how I temporarily solved my problem, until (if ever) I have the time to look deeper into the code involved or newer versions of the theme and/or the plugin solve it for good.

Just to be clear, none of the problems and troubleshooting it took to solve them can be traced back ato KB Advanced RSS alone. Atahualpa is a great but complex theme and it is their combination that turned out to be problematic. It’s not the first example of a bad mix and it won’t be the last. On the flip side, with each such incident we get to learn something new that will potentially save others and ourselves some frusrtation and time in the future.

Possibly related posts (auto generated)

Your home on camera, live

ageor — Tue, 18 Dec 2007 15:06:09 +0000

Two weeks ago I was asked again to install a couple of cameras to watch a newborn baby at home (with the sitter…).

Provided there is an ADSL in the premises, the task is trivial, tech-wise. In this case, the heart of the system is an AVM Fritz ADSL modem, 4 port switch, wireless Access Point with SIP, complemented by 2 VIVOTEK PT7137 wifi IP pan-tilt (but not zoom) VGA cameras.

The first problem in serving content from a non-static (dynamic) IP address is how to hit the correct IP, which is a moving target (I recently came across the fact that, in Greece, ISP OTEnet actually disconnects dynamic ADSL connections every 54 hours, just to change IP address!). The solution to this is dynamic DNS, in this case in the form of the free service DynDNS (other such services also exist). What dynamic DNS does is have a client program automatically update the current IP address of an ADSL connection to a host of the form .dyndns.org. In this case (lucky me), both the router and the cameras have dyndns.org support built-in. If this wasn’t the case, I could set up a dyndns.org client in a windows or linux pc to do the job. I could, alternatively, buy a static IP address, which I do not prefer for security and cost reasons. More than one dynamic DNS client can be set up for redundancy, if needed.

The other problem in serving content this way is that the Internet IP one’s router uses is different from the local IP address of this same router and each of his local network devices (yes, it’s too technical). The router takes care of back-and-forth IP translation for the outgoing connections (browsing, email reception & transmission, Instant Messaging etc.) automatically (using NAPT, Network Address Port Translation). However, when a local server needs to listen to Internet incoming traffic, this must be routed to it in a way that cannot be automated (not until UPnP, anyway, which is another story), so port translation must be manually programmed to the router and the method is called “port forwarding”. What this does is tell the router what to do with any incoming traffic, depending on the Port it uses: drop it or forward it to a different internal IP and Port, where a server is standing by.

When similar content is served by more than one servers (in this case all the IP cameras transmit audio & video using the http protocol which by default “listens” and “transmits” on port 80), we have the third problem: We need to reference two (or more) different servers serving the same service (http) using the same single external IP address.

The solution here is in the way protocol calls are made: when our browser shows it connects to http://blog.dipot.com, it is actually connecting to http://blog.dipot.com:80, where 80 represents the connection’s port number (which has possible values from 1 to 65535). The port value of 80 is implied, because it is the standard port number for the http protocol (other protocols by default, but not necessarily, connect to other ports). This essentially means that we can reference ~65000 different services (protocols) on the same IP address. So, what we do is agree between ourselves and the router to use these two (or more) addresses to call our two (or more) IP cameras (the “http://” prefix is important, because the specified ports are not usually intended for this protocol):
Camera 1: http://.dyndns.org:12001
Camera 2: http://.dyndns.org:12002
Camera n: http://.dyndns.org:1200n

So, assuming the cameras addresses are 192.168.1.21, 192.168.1.22 (& 192.168.1.2n) and all transmit to port 80 (the default) we program the router to forward calls to ports 12001, 12002 (& 1200n) to 192.168.1.21:80, 192.168.1.22:80 (& 192.168.1.2n:80). It is important for the cameras IP addresses to be static, so we don’t have another moving target to cope with.

In case the router didn’t support forwarding to a different port than the incoming, we should then program the cameras to listen and transmit to ports 12001, 12002, … 1200n, which is perfectly possible, if not simpler.

This came out larger and more technical than intended. All these, however, can be implemented in about 30′ and are applicable to other services as well.

For router-specific instructions to program port forwarding, refer to the excellent guide “Free Help Setting up Your Router or Firewall“.

Possibly related posts (auto generated)

NAS for shared storage

ageor — Thu, 29 Nov 2007 11:46:56 +0000

I have been hooked on the idea of NAS (Network Attached Storage) since around 2000, when I first read about it. Of course, capacities and costs back then were out of this world.

Today I installed an Infrant Technologies ReadyNAS NV+ with 4 Seagate ST3500630AS Barracuda 500GB SATA2 hot-swappable hard disks and 1 gigabit ethernet port. For a total cost around 1.400â‚¬, I got a 3-disk RAID5 configuration with 875 usable GB and a hot-spare disk. In plain english, 2 disks have to fail before any data is lost.

I preferred the Infrant (now owned by Netgear) solution to the much cheaper (~900â‚¬) 2 gigabit Ethernet Intel Entry Storage System SS4000-E, (also with 4 500GB SATA2 disks) for its Active Directory support. However, apart from increased peace of mind (thanks to the hot-spare) the NV+ offers some nice built-in backup and media streaming features, in addition to the 3 USB ports with support for a UPS, printers, flash and USB disks.

I am fairly sure that NAS is the way to go for shared storage. With LAN speeds, disk capacities, users and uses increasing at a hectic pace NAS is cheap (starting at <200â‚¬), scalable and very easy to setup. Please note I am not referring to server applications storage; that’s a different, much more expensive, story.