# News about next imapsync, currently distributed , next and previous releases (back to menu)

## Roadmap

If you are very interrested in those projects, don't hesitate to solicit me!

- Roadmap: Build a docker image including everything, to have a complete online /X in a single command.
- Roadmap: Or better: Include a webserver in imapsync to have an online /X integrated. (NB: proof of concept already coded and working)
- Roadmap: Add the sync of multiple accounts inside imapsync itself instead of via an external shell script doing the loop from a csv file.
- Roadmap: Build an online tool, like /X, for multiple accounts at once. (NB: proof of concept already coded and working)

## New features or bugfixes since previous releases:

imapsync was written on

See ChangeLog to read what has been done in details since 2001.

- 2.229
- General:
- Dependencies added: Proc::ProcessTable
- Dependencies removed: None
- Enhancement: Added a stat value: memory footprint times time spent during the sync. The unity is MiB * hour. The stat line is "Memory consumption at the end".
- Enhancement: Filtering buggy flags is now case insensitive. \Junk or \JUNK or \jUnK etc. (sub tests_filterbuggyflags)
- Enhancement: Added --memorystress to check memory crunching in normal run.
- Usability: Do not email final reports by default. Use --emailreport1 and --emailreport2 to add final emails reports in each INBOX.
- CGI context:
  - Refuse to serve when the cpu load is greater than 1 per cpu. A load of 3.9 with 4 cores is ok, 4.1 is not. (sub heavy_load_reached_by_cpu)
  - Refuse to serve when the RAM memory available for imapsync processes is less than 1 GB, presented by percentage on the first line like "64.80% used by processes". (sub heavy_load_percent_threshold)
  - When refuse to serve, exits with value 69 EX_UNAVAILABLE
- Testing: 1992 unit tests

- 2.200
- Dependencies removed: None
- Dependencies added: None
- Enhancement: Append a final email report on each account at the end of the synchronization. Use --noemailreport1 and --noemailreport2 to avoid final emails reports in each INBOX.
- Enhancement: Documented why total sizes can differ even when the sync is perfect.
- Enhancement: Added SERVER_NAME SERVER_ADDR SERVER_ADMIN variables and values to the output.
- Enhancement: Added the local ip address for the imap connexions. It can help configuring firewalls to allow the imap source IP.
- Enhancement: Added "Posta inviata" for \Sent with --automap
- CGI context: Upped ERRORS_MAX_CGI from 20 to 500.
- CGI context: Added --var to get values from proximapsync for variables REMOTE_ADDR REMOTE_HOST HTTP_REFERER HTTP_USER_AGENT SERVER_SOFTWARE SERVER_PORT HTTP_COOKIE
- Usability: Added warning "parsing headers of folder ... It can take time for huge folders. Be patient."
- Bug fix: Do not turn on --delete2duplicates when --syncduplicates is on, unless --delete2duplicates is given.
- Bug fix: Redirect STDERR to STDOUT in all cases, --log or --nolog
- Bug fix: Do not allow --skipcrossduplicates and --usecache. Exit EX_USAGE 64
- Bug fix: Dates were not displayed under Windows because POSIX::strftime %e is not portable. Fixed several other places where dates were wrong on Windows.
- Bug fix: --tests --testslive was listing 0 folders.
- Refactoring: Deglobalized $debuglist $debugflags
- Refactoring: Added sync->{ permanentflags2 }

- 2.178
- Dependencies removed: None
- Dependencies added: Perl module Compress::Zlib
- Enhancement: Added final statistic line: "CPU time and %cpu : 13.27 sec 8.3 %cpu 2.1 %allcpus
- Enhancement: Removed the hard limit --maxsize 35_651_584 set by --gmail2 since CAPABILITY is used and good.
- Enhancement: Does not checknoabletosearch if --justfolders
- Enhancement: When creating a folder, check if it is selectable and consider ok if it is selectable.
- Enhancement: Options --gmail1 --gmail2 together sets --useheader X-Gmail-Received --useheader Message-Id
- Enhancement: Added compression in the imap connexions. Compression is off by default no matter the capability announced. Use --compress1 and --compress2 to turn compression on.
- Enhancement: Added --keepalive1 and --keepalive2 options. Some firewalls and network gears like to timeout connections prematurely if the connection sits idle. This option enables SO_KEEPALIVE on the host1 socket. --keepalive1 is on by default since imapsync release 2.169 Use --nokeepalive1 to disable it. Same thing for --keepalive2 but for --host2. See https://metacpan.org/pod/Mail::IMAPClient#Keepalive
- Usability: Removed --fast option. It didn't do anything anyway, it was a fake option.
- Usability: Added advice inline advice on errors
  - ERR_OVERQUOTA. "The destination mailbox is 100% full, get free space on it and then resume the sync."
  - ERR_TRANSFER_EXCEEDED. "The maximum transfer size for a single sync is reached ( over 3 GB ). Relaunch the sync to sync more."
  - ERR_APPEND_SIZE. "The destination server refuses too big messages. Use --truncmess option. Read https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Too_Big.txt"
  - ERR_CONNECTION_FAILURE_HOST1. "Check that host1 on port1 is the right IMAP server to be contacted for your mailbox."
  - ERR_CONNECTION_FAILURE_HOST2. "Check that host2 on port2 is the right IMAP server to be contacted for your mailbox."
  - ERR_AUTHENTICATION_FAILURE_USER1. "Check the credentials for user1."
  - ERR_AUTHENTICATION_FAILURE_USER2. "Check the credentials for user2."
- Usability: Bytes sizes human readable are now presented in KB MB GB TB PB (1000 base) instead of KiB MiB GiB TiB PiB (1024 base)
- Usability: Added README help for --truncmess
- CGI context: Experimental loaddelay forced to 0 (no delay at all). So far a huge load is rarely a problem.
- CGI context: Bugfix. Abort didn't set charset=UTF-8; same thing for heavy load 503 Service Unavailable
- Bug fix: Made reconnect mechanism works with --oauthdirect1 --oauthdirect2 --oauthaccesstoken1 --oauthaccesstoken2
- Bug fix: Upped the maximum number a pid can be. From 99999 to 2^22. See what-is-the-maximum-value-of-the-process-id
- Bug fix: Load average on Mac was sometimes buggy. locale float 3,14 instead of 3.14
- Bug fix: errors_incr() exited with CATCH_ALL
- Bug fix: Memory consumption on Mac was not relevant. Use RSS instead of VSZ, on Mac only. Linux stays with VSZ.
- Docker context: Current working directory changed from /var/tmp to /var/tmp/uid_$EFFECTIVE_USER_ID

- 2.140
- General: New release numbering from 1.999 directly to 2.100; I want to avoid 1.1000 and sorting issues in directory listing.
- Dependencies removed: None
- Dependencies added: None
- Enhancement: Check if SEARCH ALL works on both accounts and then add --noabletosearch if one can not. Use --nochecknoabletosearch to avoid that behavior.
- Enhancement: Added an analyse and classification of the errors encountered. Most common error encountered is printed at the end. Exit value 112 (EXIT_WITH_ERRORS_MAX) is now replaced by the most common error encountered exit value:

           EX_OK          => 0  ; #/* successful termination */
           EX_USAGE       => 64 ; #/* command line usage error */
           EX_NOINPUT     => 66 ; #/* cannot open input */
           EX_UNAVAILABLE => 69 ; #/* service unavailable */
           EX_SOFTWARE    => 70 ; #/* internal software error */
           EXIT_CATCH_ALL              =>   1 ; # Any other error
           EXIT_BY_SIGNAL              =>   6 ; # Should be 128+n where n is the sig_num
           EXIT_BY_FILE                =>   7 ;
           EXIT_PID_FILE_ERROR         =>   8 ;
           EXIT_CONNECTION_FAILURE     =>  10 ;
           EXIT_TLS_FAILURE            =>  12 ;
           EXIT_AUTHENTICATION_FAILURE =>  16 ;
           EXIT_SUBFOLDER1_NO_EXISTS   =>  21 ;
           EXIT_WITH_ERRORS            => 111 ;
           EXIT_WITH_ERRORS_MAX        => 112 ;
           EXIT_OVERQUOTA              => 113 ;
           EXIT_ERR_APPEND             => 114 ;
           EXIT_ERR_FETCH              => 115 ;
           EXIT_ERR_CREATE             => 116 ;
           EXIT_ERR_SELECT             => 117 ;
           EXIT_TRANSFER_EXCEEDED      => 118 ;
           EXIT_ERR_APPEND_VIRUS       => 119 ;
           EXIT_TESTS_FAILED           => 254 ; # Like Test::More API
           EXIT_CONNECTION_FAILURE_HOST1     =>  101 ;
           EXIT_CONNECTION_FAILURE_HOST2     =>  102 ;
           EXIT_AUTHENTICATION_FAILURE_USER1 =>  161 ;
           EXIT_AUTHENTICATION_FAILURE_USER2 =>  162 ;

- Enhancement: Added option --dry1 it avoids to fetch the message on host1 in --dry mode. Option --dry1 is on when --dry is on. It speeds up --dry very much. To get the old --dry behavior, use --dry --nodry1
- Enhancement: Do both sides login and authentication before exiting on failure.
- Enhancement: --timeout can now be a float (was only integer). Same for --timeout1 --timeout2
- Enhancement: Added option --syncduplicates to sync duplicates. On by default in CGI context. Off by default on command line.
- Enhancement: Added option --trylogin. After a login failure of any kind, imapsync tries to login with the LOGIN command. Use --notrylogin to avoid that. --trylogin is on by default.
- Enhancement: Added options --oauthaccesstoken1 and --oauthaccesstoken2 to support OAUTH2 authentication.
- Enhancement: Added options --oauthdirect1 and --oauthdirect2 to support OAUTH2 authentication.
- Enhancement: Added statistic line "CPU time and %cpu". Example: "CPU time and %cpu : 9.2 sec 39.8 %cpu 19.9 %allcpus".
- Enhancement: Added option --filterbuggyflags to filter flags known to be buggy and generators of errors "BAD Invalid system flag" or "NO APPEND Invalid flag list".
- CGI context: Back from Message-Id to Message-Id Received in cgisetcontext
- CGI context: In CGI context, the log directory is no longer ./LOG_imapsync/ but simply ./ It should fasten the searches and grep
- CGI context: Added --loglogfile option. Append ../list_all_logs_auto.txt with the logfile name. On by default in CGI context.
- Usability: Changed time presentation on ETA and in final stats. Example: Wednesday 16 June 2021-06-16 00:32:05 +0200 CEST
- Usability: Do not check selectable folders for huge accounts with more than 152 folders, 98 percentile of /X; It could be very long and ended up with some users transformed in stone just by staring at the fixed output.
- Bug fix: The number of detected cpus (cores) was accurate only on Linux. Added Freebsd and fixed Darwin and Windows parts.
- Refactoring: Continuing to deglobalize the global variables.
- Refactoring: 1734 unit tests.

- 1.977
- Dependencies removed: None.
- Dependencies added: Perl modules Encode::IMAPUTF7
- Windows: imapsync.exe is now 64 bits (it was a 32bit binary before). If you use an old 32bit PC, use imapsync_32bit.exe instead.
- Enhancement: Folders sizes are now presented per folder for host1 and host2, as well as the differrences between them. Comparing folders is now easier
- Enhancement: cgidir is . when used by Net::Server::HTTP
- Enhancement: Added context in --releasecheck user_agent. CGI or Docker or Standard.
- Usability: Added ETA at the beginning of a folder process (it is also to improve the /X progress bar accuracy).
- Usability: Added the number of errors and the max number of errors allowed at the final line. Example: "Exiting with return value 0 (EX_OK: successful termination) 0/50 nb_errors/max_errors"
- Bug fix: Folders names with + characters must keep their + as they are when prensenting them in uft8 (in folders listings [] = [], right part)
- Bug fix: PIDs can be negative under Windows.
- Bug fix: --resynclabels was not on with --gmail1 --gmail2
- Bug fix: Count a fatal error as a error before presenting the count errors when exiting.
- Bug fix: Docker context. Killing myself in docker context doesn't seem to work well, as well as sending signals several times. So just install catch_exit with INT (ctr-c on terminals) and exits if still alive after killing myself.
- Bug fix: Option --justconnect did not show the line "IP address:"
- Bug fix: Reviewed all exit_clean to include a final CR on the error message.
- Bug fix: Added SSL_cipher_list => 'DEFAULT:!DH' to tests with ssl.
- Bug fix: When using --subfolder2 SUB and --delete2folders, do not delete the folder SUB (even it is not on host1).
- Bug fix: CGI context. charset was charset=ISO-8859-1 in header (CGI.pm default). Now "text/plain; charset=UTF-8"
- Bug fix:
- Refactoring: Moved main instructions into sub single_sync(). The goal is to prepare multiple different syncs in imapsync.
- Refactoring: Cgi context is now set on with env variables SERVER_SOFTWARE and SERVER_PORT (was only SERVER_SOFTWARE). It is because Net::Server::HTTP does not set SERVER_SOFTWARE but SERVER_PORT. It is a preparation to a homemade web server to serve on imapsync as a cgi and get free from Apache or a complicated web server stuff configuration.

- 1.945 (2019/06/25)
- Dependency added: Perl modules Text::ParseWords and File::Tail
- Enhancement: A new /X including:
  - Options like `--dry --justlogin --justfolders --justfoldersizes`,
  - An extra mode with `--delete1 --delete2 --subfolder1 --subfolder2` in order to make backup/restore syncs online,
  - A button to reverse source and destination parameters.
- Usability: Added the error message text after "Exiting with return value xx". Example: Exiting with return value 16 (EXIT_AUTHENTICATION_FAILURE)
- Usability: Now "imapsync --host1 imap.lamiral.info" or "imapsync --host2 127.0.0.1" or both do like a --justconnect on servers. Useful to check rapidely an imap server. Caveat: no longer a command line error.
- Usability: Added some error message texts explaining the exit numbers. Example: "Exiting with return value 16 (EXIT_AUTHENTICATION_FAILURE)"
- Usability: Added what function is called when installing a signal. Example: "kill -QUIT 19839 # special behavior: call to sub catch_exit"
- Usability: Now --justfolders activates --foldersizesatend
- Usability: Moved the warning "no header by parse_headers so taking whole header with BODY.PEEK[HEADER]" to --debug mode.
- Usability: Moved the warning "no header found so adding our own" to --debug mode.
- Bug fix: USR1 signal for toggle_sleep call was never installed.
- Bug fix: The number of messages deleted reported at the end was not accurate at all.
- Bug fix: Change the longest line in --exchange2 and --office from 10500 to 10239. (\n inserted for those long lines)
- Enhancement: Added PROCESS_ID after "Tail -f finished."
- Enhancement: --maxage and --minage can take float as value in order to filter part of a day. (Need --noabletosearch also).
- Enhancement: Gmail context. Added `--resynclabels` to resync the Gmail labels `X-GM-LABELS`.
- Enhancement: Backup/Restore context. Added `--subfolder1 "Foo/Bar"` option in order to restore a backup made with `--subfolder2 "Foo/Bar"`. The option `--folderrec "Foo/Bar"` is implied by `--subfolder1 "Foo/Bar"` because it's the most common purpose. Also, options `--subfolder1` and `--subfolder2` turn off --automap if it is set because no one would have it set in this context.
- Enhancement: All exits are now done with a specific return code related to the error causing the exit. Here is the list of the exit code values (an integer between 0 and 255), the names reflects their meaning:

           EX_OK          => 0  ; #/* successful termination */
           EX_USAGE       => 64 ; #/* command line usage error */
           EX_NOINPUT     => 66 ; #/* cannot open input */
           EX_UNAVAILABLE => 69 ; #/* service unavailable */
           EX_SOFTWARE    => 70 ; #/* internal software error */
           EXIT_CATCH_ALL              =>   1 ; # Any other error
           EXIT_BY_SIGNAL              =>   6 ; # Should be 128+n where n is the sig_num
           EXIT_PID_FILE_ERROR         =>   8 ;
           EXIT_CONNECTION_FAILURE     =>  10 ;
           EXIT_TLS_FAILURE            =>  12 ;
           EXIT_AUTHENTICATION_FAILURE =>  16 ;
           EXIT_SUBFOLDER1_NO_EXISTS   =>  21 ;
           EXIT_WITH_ERRORS            => 111 ;
           EXIT_WITH_ERRORS_MAX        => 112 ;
           EXIT_TESTS_FAILED           => 254 ; # Like Test::More API

- Usability: Gmail context. Activate --resynclabels by default with `--gmail1 --gmail2`
- Usability: The maximum size of a message is the minimum of --maxsize and APPENDLIMIT=xxx (given the imap servers) or each one if the other no exists.
- Usability: CGI context. Added default like --maxsize 1_000_000_000 (Out of memory prevention?).
- Usability: CGI context. Only use "Message-Id" to identify messages, instead of "Message-Id" + "Received". We'll see.
- Usability: CGI context. --addheader on by default. In order to remove it from the /X call. Still allow it as an option.
- Enhancement: Added option --appendlimit xxxx in order to override the value got by APPENDLIMIT in CAPABILITY. The goal is to permit --truncmess of large messages.
- Enhancement: Added option --truncmess to allow transfer of too big messages. Use it if you think it's better to have a truncated message than no message at all.
- Usability: CGI context. Instead of quit with "another imapsync may be curently running", do like a tail -f on the other running sync, if the standart (crendentials) parameters are the same.
- Usability: CGI context. A heavy load is now logged.
- Usability: CGI context. Tolerate more load. Double the threshold. Now accept load at 2 per core instead of 1 per core
- Usability: CGI context. Remove edging blanks of the 6 options `--hostX --userX --passwordX` (where X = 1 or 2). It will solve some login failures on /X just because blanks are not visible but sometimes there, copy/paste my friend you're not always my friend... use `--nosanitize` to avoid it.
- Bug fix: With `--gmail1` or `--gmail2` do not sync `[Gmail]/All Mail` if it is not requested.
- Bug fix: When asked to terminate by a signal, kill myself by the signal instead of an exit call. Applied Martin Cracauer's advice on Proper handling of SIGINT/SIGQUIT. I hope it fixes the not always working double-ctrl-c to end the sync.
- Bug fix: Fixed tests to pass `imapsync --tests` under the docker image.
- Bug fix: Gmail context. `--synclabels --subfolder2 "Foo"` create only sublabels under `"Foo"` and not a mixture with root labels.
- Bug fix: `--subfolder2 "Pf2/Sub2"` and `--subfolder2 "Sub2"` perform the same when `"Pf2"` is the prefix of host2
- Bug fix: imapsync now syncs messages whatever their proclaimed size are. (ProtonMail Bridge 1.1.3 does RFC822.SIZE null for all messages).
- Refactoring: Fixed some perl critics.
- Refactoring: Deglobalized some variables
- Security: Sanitized --subfolder1 and --subfolder2 values.
- Portability: Upgraded Mac binary to handle TLSv1.2
- Roadmap: Will add `--subfolder1` and `--subfolder2` parameters to /X in order to make backup/restore syncs.

- 1.921 (2019/02/18)
- Enhancement: Script sync_parallel_unix.sh to parallelize massive syncs. It uses the powerful GNU `parallel` command.
- Enhancement: Added option `--synclabels` to sync Gmail labels. It should speed up Gmail to Gmail syncs. The option `--synclabels` applies only during a copy of a message for now. Resyncing Gmail labels will be coded later.
- Enhancement: Option `--synclabels` is activated with `--gmail1 --gmail2` used together.
- Portability: The imapsync script Shebang (first line of imapsync) uses `env` now, instead of direct perl path. I'm not sure it's a good change that `#!/usr/bin/perl` becomes `#!/usr/bin/env perl`
- Enhancement: Final stats:
  - Added a counter of crossduplicates on host2 ( a crossduplicate is a message found accross different folders )
  - Added counter of "Messages found in host1 not in host2"
  - Added counter of "Messages found in host2 not in host1"
  - Added stat "Load end is"
  - Added comment on final stat about messages in host1 not in host2
  - Added comment on final stat about messages in host2 not in host1
  - Added comment on final stat about unidentified messages and an --addheader suggestion
  - Removed counters of "Total bytes duplicate host1" and "Total bytes duplicate host2"
  - Removed counter of "Total bytes error"
- Enhancement: Added authmech X-MASTERAUTH. To be used for Kerio with --authmech1 X-MASTERAUTH or --authmech2 X-MASTERAUTH ( Patch from https://github.com/imapsync/imapsync/pull/53/files )
- Enhancement: Added --sigprint HUP (signal HUP by default) in order to allow some asynchronous prints (with, for example, a `kill -HUP IMAPSYNC_PID`) in CGI mode and avoid timeouts from browser
- Enhancement: `--skipcrossduplicates` activated with `--gmail1` alone
- Enhancement: `--skipcrossduplicates` is NOT activated with `--gmail2` alone or with `--gmail1 --gmail2` together used.
- Enhancement: Speed. Deletions on host1 are done by batch during a resync.
- Enhancement: Added automatic `--maxsize` from `"CAPABILITY APPENDLIMIT=xxxx"` (Gmail)
- Enhancement: Added `--folderlast '[Gmail]/All Mail'` with `--gmail1 --gmail2`
- Enhancement: With `--office1` option, folder "Files" is excluded because it's a "special" folder. Like `--exclude "^Files$"`. Unless `--noexclude` is used.
- Enhancement: With `--office2` option, folder "Files" on host1 is renamed like `--f1f2 "Files=Files_renamed_by_imapsync"`. Unless `--nof1f2` is used.
- Enhancement: Added `--exclude '^Files$'` with `--office2` option.
- Enhancement: Added inline explanations for the options `--office1 --office2 --exchange1 --exchange2` (what they set and how to avoid what they set).
- Enhancement: Added `--noregexmess` to avoid the regexmess setting with `--office2` and `--exchange2` (wrap long lines >= 10500 char).
- Enhancement: Added `--noregexflag` to avoid the regexflag set by `--exchange2` (remove `\Flagged` flag).
- Enhancement: Added `IO::Socket::IP` and `CGI` in the list of the modules versions.
- Usability: Added PPID info where PID is printed. Unix only. Can be useful in some contextes, CGI, embeded, debugging etc.
- Bug fix: Now split the deletion of messages in several chunks (using --split1)
- Bug fix: --debugssl 0 was impossible and equivalent to --debugssl 1
- Bug fix: Removed `--nosubscribed` usage since it is not coded and useless anyway.
- Bug fix: Option `--maxsleep` override variable `$MAX_SLEEP` in `gmail*()` subroutines.
- Bug fix: Do not create host2 folder when no messages are found on this folder on host1 (See https://github.com/imapsync/imapsync/issues/95)
- Bug fix: The regex to remove the flag `\Flagged` was wrong. It worked because of another filter run after.
- Bug fix: The options `--exchange1` and `--exchange2` were not options! No one complained...
- Bug fix: A pid can be up to 99999 on FreeBSD
- Refactoring: Deglobarised some global variables
- Refactoring: Refactored `sub appendlimit()` to use `Mail::IMAPClient::capability` like the API. No hack from internal `$imap->{ APPENDLIMIT }`
- Refactoring: Fixed some perl critics.
- Portability: Added freebsd support for loadavg and cpu_cores.
- Portability: Calls memory_consumption_of_pids_win32() within Cygwin environment (https://github.com/imapsync/imapsync/pull/22)
- Dependency removed: None.
- Dependency added: None.
- Roadmap: Proxy mode abandoned. Will be another lighter memory tool (since less memory consumption is the goal).

- 1.882 (2018/05/06) On the road to a proxy mode! (not done yet)
- Enhancement: Added option --nocheckfoldersexist ( Default is like --checkfoldersexist )
- Enhancement: Added option --noresyncflags : Do not resync flags for already transfered messages. May be useful when a user has already started to play with its host2 account.
- Enhancement: Option --abort now creates a log too, in order to see if imapsync ended by signal comes from option --abort or something else.
- Enhancement: Added option --sigexit; For example --sigexit QUIT --sigexit TERM to exit when receiving QUIT or TERM signals.
  Added option --sigreconnect. For example --sigreconnect INT to reconnect to both imap servers when receiving INT signal.
  Added option --sigignore. For example --sigignore TERM to "ignore" the TERM signal. In fact print stats during the sync.
  Default is like: --sigexit QUIT --sigexit TERM --sigreconnect INT
- Enhancement: Moved the hostname printing at first line. The goal is for clarity when in proxy mode.
  Added free RAM info in the banner.
- Enhancement: When using sleeping behavior, a signal USR1 toggles sleep from 2s to 0s (back to 2s with on next USR1 signal). Unix only.
- Usability: Detect extra arguments. Meaning usually a quoting issue in the command line or that options are not well parsed.
- Usability: Added inline documentation about --releasecheck and --noreleasecheck Default upstream is still --releasecheck. Just a line to change to have --noreleasecheck by default (as before)
- Usability: Added inline advice to resync after an end commanded by a TERM or QUIT signal.
- Bug fix: Fixed sub epoch() to allow that kind of silly INTERNALDATE 00-Jan-0000 00:00:00 +0000 (Thanks to IBM Domino...)
- Bug fix: --releasecheck activated was hardcoded in cgi context. Now it depends on the default behavior chosen. Bug pending. Still have to fix --releasecheck on with --help
- Bug fix: Added \t\r\n characters to be replaced by character _ in sub filter_forbidden_characters()
- Bug fix: fixes to pass --tests under root user.
- CGI context: Added SERVER_SOFTWARE and SERVER_PORT info.
- CGI context: Added free RAM, total RAM and hostname in the status.
- CGI context: Reduced the loadavg info in the header status.
- Refactoring: Fixed some perlcritics level 4.
- Refactoring: Deglobalized several global variables.
- Dependency added: Perl module Regex::Common

- 1.836 (2017/09/05) More secure by default, ssl or tls activation!
- Enhancement: An Imapsync Docker image available!
- Usability: Now goes to SSL by default if port 993 is open. Use `--nosslcheck` to avoid that.
- Usability: Now goes to TLS by default if possible, ie, only if STARTTLS is in CAPABILITY. If you want only TLS and nothing else, use `--tls1 --nossl1`
- Usability: Now if you want a basic imap connection on port 143 with no default encryption behavior, ie, no ssl nor tls, then use `--nossl1 --notls1` for host1 and `--nossl2 --notls2` for host2.
- Enhancement: Added `--gmail1` and `--gmail2` to simplify Gmail options setting. It sets parameters suggested in the Gmail FAQ `--ssl`, `--host`, etc.
- Enhancement: Added `--office1` and `--office2` to simplify Office 365 options setting. It sets parameters suggested in the Exchange/Office365 FAQ.
- Enhancement: Added `--exchange1` and `--exchange2` to simplify Exchange options setting. It sets parameters suggested in the Exchange FAQ.
- Enhancement: Added `--domino1` and `--domino2` to simplify Domino options setting. It sets parameters suggested in the Domino FAQ.
- Enhancement: Added `--maxsleep` in order to avoid timeouts with `--maxbytespersecond` and `--maxmessagespersecond` options. By default imapsync will sleep 2 seconds maximum, like if the command line contained `--maxsleep 2`
- Enhancement: Added `--maxbytesafter` in order to start `--maxbytespersecond` limitation only after `--maxbytesafter` amount of data transferred. Usefull for Gmail limits, for example, in order to active a 50K/s limit rate only after 500 MB of data transfer, use `--maxbytesafter 500_000_000 --maxbytespersecond 50_000`
- Enhancement: Added `--testsunit` in order to run any unit test individualy from the command line. Several --testsunit are allowed. Example: `imapsync --testsunit tests_true --testsunit tests_always_fail`
- Enhancement: Added password setting via environment variables `IMAPSYNC_PASSWORD1` and `IMAPSYNC_PASSWORD2`
- Usability: No more useless and false warning `"says it has NO CAPABILITY for AUTHENTICATE LOGIN"`
- Usability: Options `--delete1` and `--delete` are now aliases. Option `--delete1` is preferable over `--delete` (`--delete` is still supported).
- Usability: Now prints always permanentflags info. It helps to understand most flag issues at first run, without `--debugflags`
- Usability: Now prints "could not append ( Subject:[$subject], Date:[$h1_date], Size:[$h1_size] )" when append fails.
- Usability: Option `--showpasswords` now shows also passwords with `--debugimap`. Useful to debug quoting issues.
- Usability: `--ipv4` is now synonym of `--inet4` and `--ipv6` is now synonim of `--inet6`
- Usability: Added `--testslive6` to check pure ipv6 connectivity.
- Enhancement: Added `--noabletosearch1` and `--noabletosearch2`; Still support `--noabletosearch`, which turn on both `--noabletosearch1` and `--noabletosearch2`
- Enhancement: Added `--abort` option to terminate a previous call still running. In command line context `--abort` uses the pidfile to know what to abort. In cgi context, ie online, exact same credentials are needed in order to really abort the other sync.
- Enhancement: Added milliseconds in the default logfile name since several runs is possible within one second in cgi context or on a powerful machine.
- Docker context: Added docker context in order to be run under the nobody user without permission issues.
- Docker context: Can run imapsync `--tests` under nobody user on Unix (or at least Linux).
- Bugfix: Fixed issue "SSL routines:ssl3_check_cert_and_algorithm:dh key too small" with openssl-dh-key-too-small-error `SSL_cipher_list => 'DEFAULT:!DH'`
- CGI context: Allow parameters passed by POST.
- CGI context: Abort, before doing anything, if the server load is already too heavy, and invite to come later depending on the current load (1, 5, or 15 minutes later).
- Bug fix: Guessed prefix is the empty string even when there is no folders (which is a bad sign anyway, since INBOX should be listed).
- Bug fix: Option `--skipmess` could not work most of the time. I guess it was a mistake arrived by badly converting an "unless" to an "if". Perl critics with no tests added => caveat emptor!
- Refactoring: Removed Mail::IMAPClient overload definitions
- Dependency added: IO::Socket::SSL
- Dependency added: Sys::MemInfo
- Dependency added: Pod::Usage

- 1.727 (2016/08/19) https website and CGI on the way!
- Security: Thanks to free and easy letsencrypt, Imapsync website is available in https: https://imapsync.lamiral.info/
- Enhancement: Added option `--delete1emptyfolders`. It deletes empty folders on host1, INBOX excepted. Useful with `--delete` since what remains on host1 is only what failed to be synced.
- Enhancement: Imapsync can be used as a CGI program. Still experimental. The web UI interface has to be written.
- Enhancement: XOAUTH2 authentication works with json files without extra tool openssl. Old pk12 files are still supported (still with openssl).
- Enhancement: Memory usage passed from 5x to 2x of the biggest message size. Some users do send 700MB messages, imapsync needed 3.5 GB of RAM, now it crunches only 1.4 GB, a more common RAM value available on small hosts.
- Enhancement: Added option `--inet4` to force ipv4 in ssl and tls modes. Useful with Gmail and Office365 when ipv6 is disable on the imapsync host but not ipv6 resolution.
- Enhancement: Reconnect to imap servers with a Ctrl-c. Exit with two Ctrl-c within one second or one Ctrl-\
- Enhancement: Copy is not done when one --pipemess command ends with an empty message generated.
- Enhancement: STDERR of --pipemess commands are reported on STDOUT and in the logfile (they were ignored before).
- Enhancement:
- Dependency: Added Perl module dependency of Readonly to transform hard coded magic numbers into names. Mandatory module.
- Dependency: Added Perl module dependency of Test::MockObject for some tests. Only a "require" for now.
- Security: In CGI mode, remove all content in the unsafe "eval"uated options (--regextrans2 --regexflag --regexmess --skipmess --pipemess --delete2foldersonly --delete2foldersbutnot). Consider these options not available in CGI mode.
- Bug fix: Fixed -1 wrong output in statistics for big accounts (%d => %s) in printf.
- Bug fix: Uncommented call to $test_builder->reset( ) Don't know why it was commented but it was bad for knowing tests go wrong.
- Bug fix: Check connected state in sub create_folder(). Avoid multiple useless failures.
- Refactoring: Wrote Perl package Imapsync::Getopt::Long to handle CGI requests like command line options. Started as a copy of Luke Ross Getopt::Long::CGI
- Refactoring: Perlcritic cleanup 627 violations of CodeLayout::ProhibitTrailingWhitespace.
- Refactoring: Perlcritic cleanup 60 Double-sigil dereferences.
- Refactoring: Perlcritic cleanup 458 ValuesAndExpressions::ProhibitInterpolationOfLiterals. "Useless interpolation of literal string"
- Refactoring: Perlcritic cleanup 420 CodeLayout::ProhibitParensWithBuiltins. "Builtin function called with parentheses". (Severity: 1).
- Refactoring: Perlcritic cleanup, various small advices.
- Refactoring: Replaced print by myprint, printf by myprintf, sprinf by mysprintf. Will ease `--quiet` mode and general output handling.

- 1.684 (2016/03/17) Small changes, bugfixes
- Bug fix: Imapsync.exe updated to Mail-IMAPClient-3.38 (3.37 had a bug with credentials containing special characters)
- Usability: Added inline help about "use --noexpungeaftereach to speed up" if --delete is used.
- Usability: Added --sslargs1 in documentation, usage option list.
- Bug fix: No more warning about "Use of uninitialized value $sockargs[7] in join or string at" in ssl or tls mode. Was due to "'SSL_version' => undef".
- Bug fix: No more "Can't use an undefined value as a HASH reference at /usr/bin/imapsync line 1247."
- Bug fix: Added --expungeaftereach check in sub delete_message_on_host1().

- 1.678 (2016/01/21) Improved website! (I hope...)
- Enhancement: Added `--sslargs1` and `--sslargs2` to pass any ssl parameter for host1 or host2 connection. Example: `--sslargs1 SSL_verify_mode=1 --sslargs1 SSL_version=SSLv3`
- Enhancement: Added `--timeout1 int` and `--timeout2 int` in seconds (`--timeout int` still available to set both with the same value)
- Enhancement: Added `--debugssl int`. Default is like `--debugssl 1` (Only print out errors).
- Enhancement: Added several polls to know where I shall focus improvements.
- Usability: Added env_proxy call in `sub xoauth2()` to read proxy settings from environment variable `http_proxy` without using `PERL_LWP_ENV_PROXY=1`
- Usability: Wrote TUTORIAL_Unix.
- Usability: Check f1 folder exists when specified by `--f1f2` and warns when it doesn't
- Usability: Added `--automap --justfolders --dry` in `imapsync_example.sh` and `imapsync_example.bat`
- Bug fix: Added require Encode::Byte to solve "The locale codeset (cp1252) isn't one that perl can decode" on Win32.
- Refactoring: Removed `--allow3xx` option.
- Refactoring: Continue to move to one global $sync-> in order to reduce number of parameters in routines.

- 1.670 (2015/12/03) Folders mapping made easy with --automap
- Enhancement: Added option `--automap` that guesses folders mapping, for folders like "Sent", "Junk", "Drafts", "All", "Archive", "Flagged".
  - IMAP Special-Use folders described in rfc6154 are supported and also well known names for Exchange and common clients.
  - Automap is turned off by default, to try it a safe method is to use `--automap --justautomap`, it will exit early
- Enhancement: Added `--f1f2 str1=str2` : Force folder str1 to be synced to str2.
  - Option `--f1f2` overrides any `--automap` mapping and any `--regextrans2` transformation.
  - Example `--f1f2 Spam=Junk` to map Spam folder to Junk.
- Enhancement: Added `--justautomap` to exit after seeing what will happen with --automap and --f1f2 options.
- Enhancement: Added IMAP4 QUOTA values (RFC 2087) when they are available. A warning goes up when it's time to find space.
- Enhancement: Added IMAP4 ID extension.
  - ID feature is on by default if supported by the IMAP servers, use `--noid` to turn it off.
  - ID Infos returned by the imap servers are shown.
  - Infos sent to the servers are name=imapsync, version=`imapsync --version`, os=$OSNAME`, vendor='Gilles LAMIRAL' and date=`rcs date of release`.
  - For github or other distributors you can hack this part by searching `$imapsync_id_github`
- Enhancement: Added `--logdir path` to change the default log directory. Default is `LOG_imapsync/`
- Usability: Added folders counting in outputs.
- Usability: Better output in the login part
- Usability: Guess prefixes and separators instead of forcing the user to find them.
- Usability: Output, added `Host1:` or `Host2:` at the beginning of lines.
- Bug fix: SSL_VERIFY_NONE in --ssl and --tls modes
- Bug fix: Fixed xoauth2 calls. Now XOAUTH2 Works on Windows `--authmech1 XOAUTH2` with openssl installed. See FAQ.d/FAQ.XOAUTH2.txt.
- Bug fix: Fixed IO::Socket::SSL constants issue with latest releases
- Bug fix: Changed "Host1: checking all wanted folders exist" not efficient algorithm in order to allow efficiently a 2.4 million folders account. Yes, some accounts have this.
- Refactoring: Started to use global $sync-> in order to reduce number of parameters in routines.
- Refactoring: Some perlcritic fixes.

- 1.644 (2015/07/17) Mac OS X standalone binary provided!
- Enhancement: Mac OS X standalone binary called `imapsync_bin_Darwin`
- Enhancement: Added option `--subfolder2 SUB` Move whole host1 folders hierarchy under folder SUB.
- Enhancement: Added `--fetch_hash_set "1:*"` to permit Mail2World success. Need a patched Mail::IMAPClient 3.35 in `sub fetch_hash()`
- Usability: No folders sizes if --justfolders, unless really wanted.
- Usability: No warning about messages when `--dry --justfolders` are used together.
- Bug fix: Added NOOP in --dry mode during fake APPEND in order to avoid timeouts

- 1.636 (2015/03/31)
- Enhancement: Added errors dump listing at the end, before last folder sizes or the statistics. Turned on by default. Use `--noerrorsdump` to avoid it.
- Enhancement: Added `--maxlinelengthcmd` that will be called upon when a line over `--maxlinelength` value is detected.
- Enhancement: Exchange maxlinelength issue fixed with `--maxlinelengthcmd 'reformime -r7'` on Linux. Install maildrop package to get reformime command
- Enhancement: Added `--testslive`. Running `imapsync --testslive` performs a live test on real accounts (I own).
- Enhancement: Added `--pipemess` in order to pass all message to an external filter tool like `reformime -r7`.
- Enhancement: Added xoauth2 support, available on Unix with underlying openssl. Thanks to Joaquin Lopez.
- Enhancement: Added `--skipmess regex` to skip messages matching a regex. Example `--skipmess 'm/[\x80-ff]/'` to slip messages with non-7bit characters ( ie a byte somewhere begins with 1 )
- Usability: Better output of folders excluded by --exclude and folders included by --include
- Usability: Added inline help in many places. What you can do with switches to change imapsync behaviour, contextually. Learn imapsync the easy way.
- Bug fix: `imapsync --tests` should work under any circumstance now, any Unix, Windows, exe or bin or script. 561 non-regression tests
- Bug fix: Options `--folderfirst` and `--folderlast` generated an error when their value folder does not exist. Existence is checked.
- Bug fix: Option `--disarmreadreceipts` used to change Disposition-Notification-To in the body when not available in the header. Now never changes the body in all cases.
- Bug fix: Change default useheader values. Now it is really like `--useheader "Message-Id" --useheader "Received"`. HMailServer replies two lines with `--useheader "Message-Id" --useheader "Message-ID"` in older imapsync releases.
- Bug fix: in `imap_utf7_decode()` + character must not be escaped. Was a bug with Cyrillic characters
- Refactoring: Started to split the FAQ file in several parts in FAQ.d/*
- Refactoring: Split the INSTALL file in several parts in INSTALL.d/*

- 1.607 (2014/11/14) Folders shown also in utf8.
- Caveat to upgrade on Unix!: New Unicode::String Perl module dependency, for utf8 output. See the INSTALL file.
- Enhancement: Added --skipemptyfolders to skip syncing empty host1 folders. They are not created on host2.
- Enhancement: Windows exe now uses IO::SSL 2.002 instead of 1.98 (is it really an enhancement?).
- Usability: Remove /x from regexes applied with --include and --exclude. Blanks no longer have to be explicit with \ or [ ].
- Usability: Added utf8 output for folder names, protocol native utf7imap special characters are not user friendly readable
- Usability: Moved foldersizes output after folders lists output.
- Usability: Added total size transferred after each message copied.
- Usability: After ETA, added number of total to be synced, as a denominator dddd after each message copied, like nnn/dddd where nnn is the number of messages copied. Before there was only nnn without /dddd.
- Bug fix: Cache fix on Windows. File paths with brackets [] are special, they have to be escaped with glob. For example, [Gmail] must be written [[]Gmail[]].
- Bug fix: Added several checks if IsUnconnected. Goal avoid imap commands while disconnected. Reconnexion would be better anyway.

- 1.597 (2014/10/06) Small things
- Enhancement: Added --nomixfolders to avoid merging folders that are considered different on host1 but the same on destination host2 because of case sensitivities and insensitivities.
- Bug fix: Fixed "imapsync doesn't see created folders in the listing of folder sizes at the end". (second time this bug shows up)

- 1.592 (2014/05/22) Logging by default! (an internal feature now)
- Caveat to upgrade: New IO::Tee Perl module dependency, for logging. See the INSTALL file.
- Windows upgrade: Win32 binary is now distributed in a zip file with two batches scripts ready to use it.
- Enhancement: Added logging by default in subdirectory ./LOG_imapsync/
- Enhancement: Added option --log turned on by default. Use --nolog to disable logging.
- Enhancement: Added option --logfile. Default is like ./LOG_imapsync/YYYY_MM_DD_HH_MM_SS_user2.txt where YYYY_MM_DD_HH_MM_SS is the launching date and time like 2014_05_20_23_45_10
- Enhancement: Added option --disarmreadreceipts to fix resending read receipts on host2. A must with Exchange as host2.
- Enhancement: Added FTGate support (no EXAMINE imap command used anymore).
- Usability: Updated the code to easily switch on --noreleasecheck by default but allowing --releasecheck (some users want this feature).
- Usability: Warn syncing messges will not be simulated when --dry mode is on and folder on host2 does not exist yet.
- Bug fix: Make TLS works with last IO::Socket::SSL default value (Added SSL_verify_mode => 'SSL_VERIFY_NONE') in --tls mode.
- Bug fix: Fixed bug when creating folders on host2 with --sep2=\\ (Lotus Notes).
- Bug fix: Try to get folder size when a folder on host2 has the same name but with a different case like OLD_MAIL -> old_mail. This avoids "does not exist" output.

- 1.584 (2014/02/05)
- Enhancement: Added --minmaxlinelength to select messages with long lines only. It helps to diagnostic Echange error on messages with lines longer than 9000 characters
- Enhancement: Added --debugmaxlinelength
- Bug fix: --ssl1 --tls2 was buggy because of default SSL_VERIFY_PEER. "Can not go to tls encryption on [localhost]:Unable to start TLS: Cannot determine peer hostname for verification..."
- Bug fix: Check if going to tls is ok, exit otherwise with explicit error message. Thanks to Dennis Schridde for reporting this ugly bug that deserves this CVE-2014-2014.
- Debug: Print separator given by NAMESPACE even when --sep1 oe --sep2 is used.
- Debug: Prints prefix given by NAMESPACE even when --prefix1 or --prefix2 is used. It is a preparation for advising not to use --prefix unless suggested.

Good changes made before, listed because they can help:

- Enhancement: Added --skipcrossduplicates to avoid copying messages that are already copied in another folder, good from Gmail to X.
- Enhancement: Added --debugcrossduplicates to print which messages (UIDs) are skipped with --skipcrossduplicates (and in what other folders they are).
- Enhancement: Added --folderfirst string: sync this folder first. --folderfirst "Work" --folderfirst "Friends"
- Enhancement: Added --folderlast string: sync this folder last. --folderlast "[Gmail]/All Mail" etc.
- Enhancement: Added --maxmessagespersecond to limit messages tranfer rate and fix Office365 throttle limitation (use --maxmessagespersecond 4).
- Enhancement: Added --maxbytespersecond to limit byte transfer rate.
- Enhancement: Added --debugmemory option that prints memory consumption after each message is copied.
- Enhancement: Added --authmech EXTERNAL.
- Enhancement: Adapted behavior for allowing --maxdate --mindate with --noabletosearch. Use internat date instead of Date: header.
- Enhancement: QQMail IMAP4Server success (with --noabletosearch option)
- Usability: Apply same treatment to --delete2duplicates as --delete2. If --uidexpunge2 can be done, do it, else do --expunge2 (unless --nouidexpunge2 or --noexpunge2).
- Usability: Added --timeout in the --help output. Default timeout is 120 seconds now (was nothing by imapsync, 600 with underlying modules).
- Enhancement/Bug fix: Added --ssl1_SSL_version and --ssl2_SSL_version to force the SSL_version in case the default auto-negociation does not work. Example: --ssl1 --ssl1_SSL_version SSLv3
- Enhancement: Added option --messageidnodomain to fix Mailenable bug changing the domain part of Message-Id header, and avoid duplicates
- Enhancement: Added option --syncflagsaftercopy to fix Mailenable bug not getting the flags with the APPEND
- Enhancement: Added option --maxlinelength to skip messages whose max line length is over a number of bytes. Exchange 2013 and Office365 need --maxlinelength 1000 (which is a RFC2822 must) to avoid disconnections.
- Usability: Added --fixInboxINBOX, turned on by default, to map automatically Inbox INBOX folder names.
- Usability: IMAP server removing consecutive spaces on the header part doesn't generate duplicates (MailEnable does that)
- Usability: SSL_verify_mode 0 to avoid warning about Man-In-The-Middle.
- Enhancement: Added XOAUTH authentication. Thanks to Eduardo Bortoluzzi Junior.
- Enhancement: Added --search1 and --search2 to allow different searches on each host.
- Usability: Added --delete1 as an alias for --delete
- Speed: Option --nocheckmessageexists is activated by default since --checkmessageexists often slow down transfers too much.
- Usability: Option --foldersizesatend is on if --foldersizes is on. Off if --nofoldersizesatend
- Usability: Added host2 minus host1 statistic: number of messages and bytes. One difference at the start and one at the end. Useful to detect quickly a difference between host2 and host1. Need both --foldersizes and --foldersizesatend options (which are on by default).
- Usability: Added --foldersizesatend. It gets the folders sizes at the end of the whole transfer. Turned on by default (use --nofoldersizesatend to turn it off).
- Enhancement: Added option --delete2duplicates; it deletes messages in host2 that are duplicates. --delete2duplicates is on when --delete2 is set unless --nodelete2duplicates is set too.
- Enhancement: Added option --pidfilelocking; it aborts imapsync, when just launched, in case another imapsync may be running. Turned off by default
- Usability: Added option --showpasswords. it shows passwords on output instead of "MASKED". Turned off by default. Useful to restart a run by reading the log or to see what happen with special characters.
- Bug fix: Added option --fixslash2 to avoid the 'Invalid mailbox name' error. It acts when --sep1 is character / and --sep2 is not character / and host1 folder name contains --sep2 characters. Turned on by default (--nofixslash2 to unable it).
- Enhancement: Added option --noabletosearch to allow the listing of messages without the imap "SEARCH ALL" command. It's useful for playing with poor imap servers like Softalk 7.6.4 (8.6 is fine with SEARCH ALL).
- Enhancement: Added --noexpungeaftereach to speedup --delete --expunge from Gmail.
- Enhancement: Added option --nocheckmessageexists to speed up with Tobit imap server as host1.
- Enhancement: Added option --exitwhenover option to avoid locking when transfers exceed maximum limit. See for example this Gmail constraint
  - imapsync ... --exitwhenover 2500000000 # 2.5GB if host1 is Gmail
  - imapsync ... --exitwhenover 500000000 # 500MB if host2 is Gmail
- Usability: Default headers used to identify a message are like --useheader "Message-Id" --useheader "Received" (Exchange compatibility bu default).
- Enhancement: Added option --addheader to fix the issue "ignored (no header so we ignore this message)" (on Sent folder usually). Thanks to Ameir Abdeldayem
- Enhancement: Added --nofilterflags to check any STORE flags by ignoring PERMANENTFLAGS (Exchange tests)

This document last modified on ($Id: news.shtml,v 1.87 2022/09/15 08:42:40 gilles Exp gilles $)
Top of the page