123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487 |
- <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
- <HTML>
- <HEAD>
- <META NAME="generator" CONTENT="http://txt2tags.org">
- <TITLE>Imapsync tutorial</TITLE>
- </HEAD><BODY BGCOLOR="white" TEXT="black">
- <CENTER>
- <H1>Imapsync tutorial</H1>
- <FONT SIZE="4"><I>Gilles LAMIRAL gilles@lamiral.info</I></FONT><BR>
- <FONT SIZE="4">$Id: TUTORIAL_Unix.t2t,v 1.27 2022/01/14 12:42:51 gilles Exp gilles $ </FONT>
- </CENTER>
- <P></P>
- <HR NOSHADE SIZE=1>
- <P></P>
- <UL>
- <LI><A HREF="#toc1">1. Good practices overview</A>
- <LI><A HREF="#toc2">2. Basic steps</A>
- <UL>
- <LI><A HREF="#toc3">2.1. Verifying imapsync works well</A>
- <LI><A HREF="#toc4">2.2. Working with your data</A>
- <LI><A HREF="#toc5">2.3. Prepare your credentials</A>
- <LI><A HREF="#toc6">2.4. Take a real user account as a source</A>
- <LI><A HREF="#toc7">2.5. Take a test user account as a destination</A>
- <LI><A HREF="#toc8">2.6. Edit your own script mysync</A>
- </UL>
- <LI><A HREF="#toc9">3. Background knowledge about mailboxes</A>
- <LI><A HREF="#toc10">4. Imapsync presentation</A>
- <LI><A HREF="#toc11">5. Conventions</A>
- <LI><A HREF="#toc12">6. Why start with a test account on destination host2?</A>
- <LI><A HREF="#toc13">7. Imapsync default behavior</A>
- <LI><A HREF="#toc14">8. To go further with imapsync</A>
- </UL>
- <P></P>
- <HR NOSHADE SIZE=1>
- <P></P>
- <A NAME="toc1"></A>
- <H1>1. Good practices overview</H1>
- <UL>
- <LI>Do the basic checks showing imapsync works by itself where you run it.
- <P></P>
- <LI>Next, applying imapsync to your data,
- continue with a <B>real user</B> account on the source imap server (host1)
- and a <B>test</B> account on the destination imap server (host2).
- <P></P>
- It's the best advice I can give to learn imapsync, be confident in it, and
- verify it will do what you expect it to do in your context, all of that
- without much pain. So try imapsync with a real account at the source, aka host1,
- and a test account at the destination, aka host2.
- <P></P>
- <LI>Next, once familiar and satisfied by the result on the host2 test account,
- change to a real user account on host2 or just stop considering it a test one.
- </UL>
- <A NAME="toc2"></A>
- <H1>2. Basic steps</H1>
- <A NAME="toc3"></A>
- <H2>2.1. Verifying imapsync works well</H2>
- <P>
- Open a terminal and go to the imapsync directory.
- The imapsync directory is the directory created by the extraction
- of the tarball (.tgz), its name is <CODE>imapsync-2.xxx</CODE> where <CODE>2.xxx</CODE>
- is imapsync release number (2.178 at the time of this proofread).
- </P>
- <PRE>
- cd imapsync-2.xxx/
- </PRE>
- <P>
- Verify imapsync runs on your system
- </P>
- <PRE>
- ./imapsync
- </PRE>
- <P>
- It should output the help message.
- </P>
- <P>
- If the previous command fails then there is an installation issue.
- Go back to the <A HREF="https://imapsync.lamiral.info/#install">https://imapsync.lamiral.info/#install</A> section, then
- read and apply the installation file corresponding to your
- system and drop me an email about your issue.
- </P>
- <P>
- Next, verify imapsync runs well with live tests. This check needs Internet
- access. It does a simple sync between two real dedicated
- imap mailboxes located at the host test.lamiral.info
- </P>
- <PRE>
- ./imapsync --testslive
- </PRE>
- <P>
- Now verify that the script examples/imapsync_example.sh runs fine:
- </P>
- <PRE>
- sh examples/imapsync_example.sh
- </PRE>
- <P>
- This script does the same thing as "<CODE>imapsync --testslive</CODE>" but it
- uses explicitly the 6 parameters so it will be a good start
- for your future own script.
- </P>
- <A NAME="toc4"></A>
- <H2>2.2. Working with your data</H2>
- <P>
- I consider you're still in the imapsync top directory.
- </P>
- <P>
- Make a copy of the script <CODE>examples/imapsync_example.sh</CODE>
- </P>
- <PRE>
- cp examples/imapsync_example.sh mysync
- </PRE>
- <P>
- Check that the copy works as the original
- </P>
- <PRE>
- sh mysync
- </PRE>
- <P>
- So far so good, now we're going to work with your data.
- </P>
- <A NAME="toc5"></A>
- <H2>2.3. Prepare your credentials</H2>
- <P>
- An IMAP account is accessed with 3 parameters,
- </P>
- <UL>
- <LI>the imap server <B>host</B>. It's a server name or an IP address
- <LI>the <B>user</B> name
- <LI>the <B>password</B>
- </UL>
- <P>
- Since imapsync job is to sync two imap accounts, we need 3 + 3 = 6 parameters:
- </P>
- <UL>
- <LI>Three parameters to read data from the source account: <B>host1</B>, <B>user1</B> and <B>password1</B>
- <LI>Three parameters to copy this data to the destination account: <B>host2</B>, <B>user2</B> and <B>password2</B>
- </UL>
- <A NAME="toc6"></A>
- <H2>2.4. Take a real user account as a source</H2>
- <P>
- Even during the learning time with imapsync, you can take a
- real user account as a source. There is also no problem if this account is
- currently used by a user. By default, this account will only be read, no change will
- be made by imapsync on it. It's safe to use a normal and live account as a source, even to learn imapsync.
- </P>
- <P>
- Assuming that the imap source server name host1 is <B>origin.example.com</B>,
- the user1 account name is <B>myuser1</B> and its password is <B>mysecret1</B>,
- we now have the first three parameters.
- </P>
- <UL>
- <LI>--host1 <B>origin.example.com</B>
- <LI>--user1 <B>myuser1</B>
- <LI>--password1 <B>mysecret1</B>
- </UL>
- <A NAME="toc7"></A>
- <H2>2.5. Take a test user account as a destination</H2>
- <P>
- Unlike the source side, the destination side will be modified by
- imapsync. Therefore, for learning, checking, and adjusting,
- it is not a good idea to use a real user imap account
- the first time you play with imapsync.
- </P>
- <P>
- If you really can't afford a test account on host2, it's ok,
- imapsync is not that bad but you may have some work to do to fix some unwanted behavior.
- Unwanted behavior is mostly folders names that you don't want to be the same on both sides.
- </P>
- <P>
- Assuming that the imap destination server name host2 is <B>destiny.example.com</B>,
- the user2 account name is <B>myuser2</B> and its password is <B>mysecret2</B>,
- we now have the next three parameters.
- </P>
- <UL>
- <LI>--host2 <B>destiny.example.com</B>
- <LI>--user2 <B>myuser2</B>
- <LI>--password2 <B>mysecret2</B>
- </UL>
- <A NAME="toc8"></A>
- <H2>2.6. Edit your own script mysync</H2>
- <P>
- Now edit the script <CODE>mysync</CODE> and replace the test values with yours.
- </P>
- <P>
- You're ready for a dry test on your accounts.
- </P>
- <PRE>
- sh mysync
- </PRE>
- <P>
- Since the <CODE>mysync</CODE> script is a copy of <CODE>examples/imapsync_example.sh</CODE>,
- your first run with your data should include three other options
- <CODE>--automap</CODE> <CODE>--justfolders</CODE> <CODE>--dry</CODE>.
- With <CODE>--dry</CODE> option, nothing will be done for real on host2
- but yet it will test whether the credentials are ok on both sides.
- You'll encounter two successful logins, or one login failure on host1 or host2, or two failures on host1 and host2.
- If the logins are ok, you will be able to observe that the folders mapping is ok.
- </P>
- <P>
- If login fails then double-check all three values that identify
- the account, which are the host, the login name, and the password.
- </P>
- <P>
- If the folders mapping proposed is not ok then you can fix it with
- the option <CODE>--f1f2</CODE>. The following example maps the source folder
- "Sent Messages" to the destination folder "Sent". The double-quotes
- are not part of the names of the folders but they should be used when special
- characters like blanks are in the names of the folders:
- </P>
- <PRE>
- ./imapsync ... --f1f2 "Sent Messages=Sent"
- </PRE>
- <P>
- As explained in the inline help or the README:
- </P>
- <PRE>
- --f1f2 str1=str2 : Force folder str1 to be synced to str2.
- </PRE>
- <P>
- You're ready for a real test on your accounts, restricted to
- folders. Remove <CODE>--dry</CODE> from the <CODE>mysync</CODE> script and rerun <CODE>mysync</CODE>:
- </P>
- <PRE>
- sh mysync
- </PRE>
- <A NAME="toc9"></A>
- <H1>3. Background knowledge about mailboxes</H1>
- <P>
- Three Internet protocols are used to access almost all email accounts:
- POP3, IMAP, HTTP.
- </P>
- <P>
- The oldest protocol still used to access mailboxes is POP3, the Post Office Protocol.
- POP3 gives access to only one main box called INBOX.
- </P>
- <P>
- With POP3, messages have no flags at all, no Seen/UnSeen, Forwarded, or Flagged labels.
- </P>
- <P>
- It's not systematic but messages are often removed from a POP3 server
- each time a software client looks into it,
- so messages only appear on the client host that fetched them,
- they are unavailable from any other system located elsewhere.
- </P>
- <P>
- The second protocol to deal with email messages is IMAP, Internet Message Access Protocol.
- IMAP gives access to a hierarchy of mailboxes also called folders. Other IMAP features are
- concurrent accesses, tagging with flags, search by many criteriums like date, subject, size, etc.
- </P>
- <P>
- The IMAP protocol presents most of the features POP lacks.
- Messages stay on the imap server so any client on the network can access them
- at any time from anywhere, the same messages with the same flags.
- </P>
- <P>
- The third protocol to access email messages is HTTP, HyperText Transfer Protocol.
- HTTP is the protocol to browse the web.
- </P>
- <P>
- Web browsers like Google Chrome, Mozilla Firefox, Internet Explorer and Safari
- are HTTP client software tools. You already know that so what's the point with HTTP mailboxes?
- HTTP mailboxes are called webmails.
- Webmails often offer the same features as imap servers
- because webmails underlying storage systems are often imap servers.
- </P>
- <P>
- Webmails systems like Gmail, Yahoo, Exchange, Zimbra, or Office365 are also accessible via imap.
- </P>
- <P>
- The conclusion of this protocol review is that mailboxes can be
- accessed using the IMAP protocol, most of the time.
- Here comes imapsync.
- </P>
- <P>
- In case the source mailbox is only accessible by the POP protocol,
- you can use the tool <CODE>pop2imap</CODE> to sync them.
- <CODE>pop2imap</CODE> is located at <A HREF="http://www.linux-france.org/prj/pop2imap/">http://www.linux-france.org/prj/pop2imap/</A>
- </P>
- <A NAME="toc10"></A>
- <H1>4. Imapsync presentation</H1>
- <P>
- Software imapsync is a command-line tool to
- copy, migrate, backup, or synchronize IMAP mailboxes.
- </P>
- <P>
- Command line means imapsync is not graphical, it is textual.
- Usually, with command-line tools, you have to type characters
- on your keyboard. But your fingers won't suffer much pain
- typing on the keyboard because script examples are given,
- nearly ready to run. Most of the time you only have to change
- the main values in those files and adapt them to your context.
- </P>
- <P>
- Don't be afraid, the mouse won't be forsaken.
- You can still use the mouse to launch an editor,
- select/copy/paste complete examples,
- and run the little script with a double-click.
- </P>
- <P>
- Imapsync runs on Linux, Windows, and OS X (Macintosh/Darwin world).
- Imapsync is written in the Perl language and, thanks to the
- Perl developers, Perl runs mostly everywhere. So does imapsync.
- </P>
- <P>
- While operating systems have a lot in common, they sometimes differ,
- especially within their syntax. I won't blame anyone, historically Windows
- came after Unix. The marvelous designers in these old times
- decided it would be very cool to not share the same syntax
- for doing the same things.
- </P>
- <P>
- To designate an end of the line, Unix uses the character \n
- Windows uses two characters \r\n and Mac use \r.
- Thanks to you guys, great thinking!
- Fifty years later, we still suffer from this...daily.
- </P>
- <P>
- To avoid some headaches with systems that no one masters
- I will give examples in both worlds, Unix and Windows.
- OS X users are in the Unix world nowadays so they must follow
- the Unix examples.
- </P>
- <A NAME="toc11"></A>
- <H1>5. Conventions</H1>
- <P>
- To simplify display or print,
- each imapsync command line is usually written in several lines
- but it could be written in one single line.
- </P>
- <P>
- If you prefer to use the whole command written in one single line then
- just remove the last visible character of each line ( \ or ^ ) and
- also the carriage return character.
- The last visible character \ or ^ means "command continues on next line";
- it's the backslash \ character on Unix and the caret ^ character on Windows.
- </P>
- <P>
- For example, on Unix, a command like the following:
- </P>
- <PRE>
- imapsync \
- --host1 test.lamiral.info \
- --user1 test1 \
- --password1 secret1 \
- ...
- </PRE>
- <P>
- is equivalent to:
- </P>
- <PRE>
- imapsync --host1 test.lamiral.info --user1 test1 --password1 secret1 ...
- </PRE>
- <P>
- and on Windows, a command like the following:
- </P>
- <PRE>
- imapsync.exe ^
- --host1 test.lamiral.info ^
- --user1 test1 ^
- --password1 secret1 ^
- ...
- </PRE>
- <P>
- is equivalent to:
- </P>
- <PRE>
- imapsync --host1 test.lamiral.info --user1 test1 --password1 secret1 ...
- </PRE>
- <A NAME="toc12"></A>
- <H1>6. Why start with a test account on destination host2?</H1>
- <P>
- A little explanation about this hint.
- Imapsync is safe with accounts on host1,
- it doesn't change anything on them, it just read them.
- The exception of this safe principle is when <CODE>--delete1</CODE> option is used,
- since <CODE>--delete1</CODE> removes on host1 each message successfully copied to host2,
- messages that couldn't be transferred stay on host1.
- </P>
- <P>
- It's not the same for destination accounts as imapsync writes on host2 accounts.
- Imapsync creates folders on them, adds messages, and sets flags on messages.
- It isn't safe on a real account. So don't use a real user account
- to test imapsync. Learn to use it and see what it does on a test account at host2.
- </P>
- <P>
- What can badly happen? The most common bad behavior is
- that folders mapping won't be what you expect because it is strictly
- reproduced from host1 to host2. The second bad behavior is getting
- duplicates on the second run and after; it's rare but it can happen
- when an imap server software changes the headers "<CODE>Message-Id</CODE>" or "<CODE>Received</CODE>".
- Solutions to avoid duplicates are often easy (There's a FAQ called <CODE>FAQ.Duplicates.txt</CODE> about that).
- It's also possible to remove the duplicates on host2 but it's better to avoid them on user accounts at first,
- users won't like that you mess up their mailboxes.
- </P>
- <A NAME="toc13"></A>
- <H1>7. Imapsync default behavior</H1>
- <P>
- By default, unless explicitly told to do something else:
- </P>
- <UL>
- <LI>Imapsync <B>goes ssl or tls</B> if possible
- <LI>Imapsync syncs <B>all folders</B> of host1
- <LI>Imapsync syncs <B>all messages</B> from host1, except duplicates.
- <LI>Imapsync doesn't sync already synced messages.
- <LI>Imapsync syncs <B>all flags</B>, at least all allowed by host2.
- <LI>Imapsync resyncs flags on messages already synced.
- </UL>
- <A NAME="toc14"></A>
- <H1>8. To go further with imapsync</H1>
- <P>
- Imapsync has many options but you can ignore most of them
- and still make great transfers.
- </P>
- <UL>
- <LI>Option names all begin with two minus characters <CODE>--</CODE>, like <CODE>--automap</CODE> or <CODE>--dry</CODE> etc. (using one minus, like <CODE>-dry</CODE>, is ok)
- <LI>Option names relative to the <B>source</B> account are ended with the number one <B><CODE>1</CODE></B>, like in <CODE>--host1</CODE>
- <LI>Option names relative to the <B>destination</B> account are ended with the number two <B><CODE>2</CODE></B>, like in <CODE>--host2</CODE>
- <LI>Some options need a <B>value</B> just after them, like <CODE>--host1</CODE> <B><CODE>source.example.com</CODE></B>, (the value is <CODE>source.example.com</CODE>)
- <LI>Some options are standalone, like <B><CODE>--automap</CODE></B>
- <LI>Any order is possible but when an option needs a value then the value must follow immediately its option name.
- </UL>
- <!-- html code generated by txt2tags 2.6 (http://txt2tags.org) -->
- <!-- cmdline: txt2tags -i doc/TUTORIAL_Unix.t2t -t html -\-toc -o doc/TUTORIAL_Unix.html -->
- </BODY></HTML>
|