IMAPsync/FAQ.d/FAQ.Flags.txt

#!/bin/cat
$Id: FAQ.Flags.txt,v 1.28 2022/06/22 17:07:35 gilles Exp gilles $

This documentation is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Flags.txt

=======================================================================
               Imapsync tips about flags.
=======================================================================

Questions answered here are:

Q. How to debug flag issues?

Q. Is there a way to only sync messages with a specific flag set,
   for example, the \Seen flag?

Q. How to convert flags?

Q. Exchange sends an email to any sender whose email is deleted 
   without reading. It's called "unread notifications".
   How to set the \Seen flag on host1 (source system) before syncing?

Q. Every single mail synced to exchange online owns the 
   category $MDNSent after migration. How can I avoid this?

Q. Does imapsync retain the \Answered and $Forwarded flags?

Q. How to fix this error: BAD Invalid system flag \FORWARDED
   
Q. How to convert flags with $ to \ character?

Q. imapsync fails with the following error:
flags from : [\Seen NonJunk]
Error trying to append string: 58 NO APPEND Invalid flag list   

Q. Flags are not well synchronized. Is it a bug?

Q. Flags are resynced at each run for already synced/copied messages, 
   how can I avoid this feature?

Q. Is it possible to sync labels and stars made by Thunderbird to
   Exchange categories? Or a way in Outlook to show labels created by
   Thunderbird?

Now the questions again with their answers.

=======================================================================
Q. How to debug flag issues?

R. Use --debugflags

  imapsync ... --debugflags


=======================================================================
Q. Is there a way to only sync messages with a specific flag set,
   for example, the \Seen flag?

R. use --search 

  imapsync ... --search SEEN

or

  imapsync ... --search UNSEEN

or ...

The complete list of search things related to flags are listed below

http://www.faqs.org/rfcs/rfc3501.html

6.4.4.  SEARCH Command
...
      ANSWERED
         Messages with the \Answered flag set.

      DELETED
         Messages with the \Deleted flag set.

      DRAFT
         Messages with the \Draft flag set.

      FLAGGED
         Messages with the \Flagged flag set.

      KEYWORD <flag>
         Messages with the specified keyword flag set.

      NEW
         Messages that have the \Recent flag set but not the \Seen flag.
         This is functionally equivalent to "(RECENT UNSEEN)".

      NOT <search-key>
         Messages that do not match the specified search key.

      OLD
         Messages that do not have the \Recent flag set.  This is
         functionally equivalent to "NOT RECENT" (as opposed to "NOT
         NEW").

      OR <search-key1> <search-key2>
         Messages that match either search key.

      RECENT
         Messages that have the \Recent flag set.

      SEEN
         Messages that have the \Seen flag set.

      UNANSWERED
         Messages that do not have the \Answered flag set.

      UNDELETED
         Messages that do not have the \Deleted flag set.

      UNDRAFT
         Messages that do not have the \Draft flag set.

      UNFLAGGED
         Messages that do not have the \Flagged flag set.

      UNKEYWORD <flag>
         Messages that do not have the specified keyword flag set.

      UNSEEN
         Messages that do not have the \Seen flag set. 

=======================================================================
Q. How to convert flags?

R. use --regexflag
For example to convert flag IMPORTANT to flag CANWAIT

  imapsync ... --regexflag "s/IMPORTANT/CANWAIT/g" --debugflags

option --debugflags is usefull to see in details what imapsync
does with flags.


=======================================================================
Q. Exchange sends an email to any sender whose email is deleted 
   without reading. It's called "unread notifications".
   How to set the \Seen flag on host1 (source system) before syncing?

R. You can add \Seen (if missing) to the host1 account by applying a 
   first sync to the same account, same at source and destination, 
   and the help of option --regexflag. It can also be done on the fly
   from account1 to account2 in case account1 has to stay as it is.
   
Add the \Seen flag to all messages like this:

On Winwows:

  imapsync.exe ... --regexflag "s,\\Seen,," --regexflag "s,,\\Seen ,"
                   
On Unix:

  imapsync ... --regexflag 's,\\Seen,,' --regexflag 's,,\\Seen ,'

R2. You can also filter with --search1 UNSEEN and use only one
    of the regexes:

On Unix:

  imapsync ... --search1 UNSEEN --regexflag 's,,\\Seen ,'

On Winwows:

  imapsync.exe ... --search1 UNSEEN  --regexflag "s,,\\Seen ,"

R3. Fix it on the server Exchange:
Google translate: 
https://translate.google.com/translate?sl=auto&tl=en&u=https%3A%2F%2Fwww.ci-solution.com%2Fblog%2Fartikel%2Fungelesen-geloescht-verhindern.html
German original:
https://www.ci-solution.com/blog/artikel/ungelesen-geloescht-verhindern.html
(Link from Oliver B.)

======================================================================
Q. Every single mail synced to exchange online owns the 
   category $MDNSent after migration. How can I avoid this?

R. To remove the flag $MDNSent from all messages, use:

  imapsync ... --regexflag "s/\$MDNSent//g"

See also the document
https://imapsync.lamiral.info/FAQ.d/FAQ.Flags.txt

=======================================================================
Q. Does imapsync retain the \Answered and $Forwarded flags?

R. It depends on the destination server.

a) If the destination server honours the "PERMANENTFLAGS \*" 
directive (meaning it accepts any flag) or no PERMANENTFLAGS at all 
then imapsync synchronizes all flags except the \Recent flag
(RFC 3501 says about \Recent flag "This flag can not be 
altered by the client.").

b) If the destination server honours the "PERMANENTFLAGS without the 
special "\*"  then imapsync synchronizes only the flags listed 
in PERMANENTFLAGS.

Some imap servers have problems with flags not beginning with
the backslash character \ 
(see next question to find a solution to this issue)


=======================================================================
Q. How to fix this error: BAD Invalid system flag \FORWARDED

R. Filter flag \FORWARDED with --regexflag like this:

On Windows:

  imapsync.exe ... --regexflag "s/\\FORWARDED//gi" 

On Unix:

  imapsync ... --regexflag 's/\\FORWARDED//gi' 

or (on Unix)

  imapsync ... --regexflag "s/\\\\FORWARDED//gi" 

Other related flags to remove are \Indexed and \ATTACHED

Windows:   imapsync.exe ... --regexflag "s/\\Indexed//gi" 
Unix:      imapsync ...     --regexflag 's/\\Indexed//gi' 

Windows:   imapsync.exe ... --regexflag "s/\\ATTACHED//gi" 
Unix:      imapsync ...     --regexflag 's/\\ATTACHED//gi' 

Removing usually problematic flags in one line:

Windows:   imapsync.exe ... --regexflag "s/\\FORWARDED|\\Indexed|\\ATTACHED|\\RECEIPTCHECKED//gi" 
Unix:      imapsync     ... --regexflag 's/\\FORWARDED|\\Indexed|\\ATTACHED|\\RECEIPTCHECKED//gi' 

Removing all problematic flags I've seen so far in one line:
Windows:   imapsync.exe ... --regexflag "s/\\RECEIPTCHECKED|\\Indexed|\\X-EON-HAS-ATTACHMENT|\\UNSEEN|\\ATTACHED|\\X-HAS-ATTACH|\\FORWARDED|\\FORWARD|\\X-FORWARDED|\\\$FORWARDED|\\PRIORITY|\\READRCPT//g"
Unix:      imapsync     ... --regexflag 's/\\RECEIPTCHECKED|\\Indexed|\\X-EON-HAS-ATTACHMENT|\\UNSEEN|\\ATTACHED|\\X-HAS-ATTACH|\\FORWARDED|\\FORWARD|\\X-FORWARDED|\\\$FORWARDED|\\PRIORITY|\\READRCPT//g'

Removing all flags but the standard ones from the IMAP RFC:
Windows:   imapsync.exe ... --regexflag "s/.*?(?:(\\(?:Answered|Flagged|Deleted|Seen|Draft)\s?)|$)/defined($1)?$1:q()/eg"
Unix:      imapsync     ... --regexflag 's/.*?(?:(\\(?:Answered|Flagged|Deleted|Seen|Draft)\s?)|$)/defined($1)?$1:q()/eg'

=======================================================================
Q. How to convert flags with $ to \ character?

R. $ and \ are special characters we have to "escape" them.
For example to convert flag $label1 to \label1

  imapsync ... --regexflag "s/\$label1/\\label1/g" --debugflags


======================================================================
Q. imapsync fails with the following error:
flags from : [\Seen NonJunk]
Error trying to append string: 58 NO APPEND Invalid flag list

R. For some servers, flags have to begin with a \ character.
The flag "NonJunk" may be a invalid flag for your server 
so use for example:

imapsync ...    --regexflag "s/NonJunk//g"

Remark (thanks to Arnt Gulbrandsen): 
IMAP system flags have to begin with \ character.
Any other flag must begin with another character.
System flags are just flags defined by an RFC instead of by users.
Conclusion, some imap server coders don't read the RFCs (so do I).

Recent imapsync deals with this issue by filter with PERMANENTFLAGS
automatically.

=======================================================================
Q. Flags are not well synchronized. Is it a bug?

R. It happens with some servers on the first sync. 
Also, it was a bug from revision 1.200 to revision 1.207

Two solutions: 

* Run imapsync a second time. imapsync synchronizes flags on each run.

* Use option --syncflagsaftercopy. With this option imapsync will 
  also sync flags after each message transfer. Flags are already 
  synced during the transfer with the imap APPEND command but 
  option --syncflagsaftercopy does it again using the imap STORE 
  command.

=======================================================================
Q. Flags are resynced at each run for already synced/copied messages, 
   how can I avoid this feature?

R. Use the option --noresyncflags


=======================================================================
Q. Is it possible to sync labels and stars made by Thunderbird to
   Exchange categories? Or a way in Outlook to show labels created by
   Thunderbird?

R. Imapsync syncs all flags possible by default, so if it doesn't do 
that it might means there are not on the server but stay only on the 
client or that the host2 server claims to accept only a given set 
of flags.

Check those claims by accessing the same mailbox on the same server 
from another thunderbird on another host, you should not retrieve
those labels. If you do find them it then might mean that host2 
server don't want them, try --nofilterflags

  imapsync ... --nofilterflags

You can try --nofilterflags straightaway without the 
"other thunderbird" proposal.

There is also the possibility to map flags across servers 
with the option --regexflag. See some --regexflag examples 
above. 


=======================================================================
=======================================================================