[maildropl] Getting xfilter AND delivery to working

[Philip Rhoades]

People,

I have this filter:

if( /^[TC][[co]:.*tst/ )
{
xfilter "/home/phil/bin/bash_internal_variables.sh > /home/phil/t"
to $DEFAULT/.tst/
}

and the xfilter is working but the delivery is not - if I reverse the
lines, the delivery works but the xfilter does not - how can I get BOTH
lines to work?

Thanks,

Phil.

[Philip Rhoades]

Milan,

On Thu, 04 Feb 2016 19:14:24 +1100

Post by Philip Rhoades
People,
if( /^[TC][[co]:.*tst/ )
{
xfilter "/home/phil/bin/bash_internal_variables.sh

/home/phil/t" to $DEFAULT/.tst/

}
and the xfilter is working but the delivery is not - if I reverse the
lines, the delivery works but the xfilter does not - how can I get
BOTH lines to work?

Hi,
did zou examine your mailogs? Maybe there is a clue...

No errors or anything unusual . .

Also, could you
show us what's there in /home/phil/bin/bash_internal_variables.sh file?

#!/bin/bash

echo -n "! | " ; echo $!
echo -n "# | " ; echo $#
echo -n "$ | " ; echo $$
echo -n "* | " ; echo $*
echo -n "- | " ; echo $-
echo -n "0 | " ; echo $0
echo -n "1 | " ; echo $1
echo -n "2 | " ; echo $2
echo -n "? | " ; echo $?
echo -n "@ | " ; echo $@
echo -n "BASH | " ; echo $BASH
echo -n "BASHPID | " ; echo $BASHPID
echo -n "BASH_ENV | " ; echo $BASH_ENV
echo -n "BASH_SUBSHELL | " ; echo $BASH_SUBSHELL
echo -n "BASH_VERSINFO[n] | " ; echo $BASH_VERSINFO[n]
echo -n "BASH_VERSION | " ; echo $BASH_VERSION
echo -n "CDPATH | " ; echo $CDPATH
echo -n "DIRSTACK | " ; echo $DIRSTACK
echo -n "EDITOR | " ; echo $EDITOR
echo -n "ENV | " ; echo $ENV
echo -n "EUID | " ; echo $EUID
echo -n "FUNCNAME | " ; echo $FUNCNAME
echo -n "GLOBIGNORE | " ; echo $GLOBIGNORE
echo -n "GROUPS | " ; echo $GROUPS
echo -n "HOME | " ; echo $HOME
echo -n "HOSTNAME | " ; echo $HOSTNAME
echo -n "HOSTTYPE | " ; echo $HOSTTYPE
echo -n "IFS | " ; echo $IFS
echo -n "IFS defaults to | " ; echo $IFS defaults to
echo -n "IGNOREEOF | " ; echo $IGNOREEOF
echo -n "LC_COLLATE | " ; echo $LC_COLLATE
echo -n "LC_CTYPE | " ; echo $LC_CTYPE
echo -n "LINENO | " ; echo $LINENO
echo -n "LOGNAME | " ; echo $LOGNAME
echo -n "MACHTYPE | " ; echo $MACHTYPE
echo -n "MAIL | " ; echo $MAIL
echo -n "OLDPWD | " ; echo $OLDPWD
echo -n "OSTYPE | " ; echo $OSTYPE
echo -n "PATH | " ; echo $PATH
echo -n "PIPESTATUS | " ; echo $PIPESTATUS
echo -n "PIPESTATUS[0] | " ; echo $PIPESTATUS[0]
echo -n "PIPESTATUS[1] | " ; echo $PIPESTATUS[1]
echo -n "PPID | " ; echo $PPID
echo -n "PROMPT_COMMAND | " ; echo $PROMPT_COMMAND
echo -n "PS1 | " ; echo $PS1
echo -n "PS2 | " ; echo $PS2
echo -n "PS3 | " ; echo $PS3
echo -n "PS4 | " ; echo $PS4
echo -n "PWD | " ; echo $PWD
echo -n "REPLY | " ; echo $REPLY
echo -n "SECONDS | " ; echo $SECONDS
echo -n "SHELL | " ; echo $SHELL
echo -n "SHELLOPTS | " ; echo $SHELLOPTS
echo -n "SHLVL | " ; echo $SHLVL
echo -n "TERM | " ; echo $TERM
echo -n "TMOUT | " ; echo $TMOUT
echo -n "UID | " ; echo $UID
echo -n "USER | " ; echo $USER
echo -n "USERNAME | " ; echo $USERNAME
echo -n "_ | " ; echo $_
echo -n "time | " ; echo $time

but why should anything in there matter? Even if an error code was
returned, maildrop should still execute the "to" command?

I am using xfilter in various ways in the same recipe with (mostly) cc,
but to works as well, so I think your issue could be something else.
And, last but not least, you did not tell us which version are you
using.

maildrop-2.8.1-3.fc23.x86_64

Thanks,

Phil.

[Philip Rhoades]

Milan,

On Fri, 05 Feb 2016 00:34:36 +1100

Post by Philip Rhoades
Milan,

On Thu, 04 Feb 2016 19:14:24 +1100

Post by Philip Rhoades
People,
if( /^[TC][[co]:.*tst/ )
{
xfilter "/home/phil/bin/bash_internal_variables.sh

/home/phil/t" to $DEFAULT/.tst/

}
and the xfilter is working but the delivery is not - if I reverse
the lines, the delivery works but the xfilter does not - how can I
get BOTH lines to work?

Hi,
did zou examine your mailogs? Maybe there is a clue...

No errors or anything unusual . .

Also, could you
show us what's there in /home/phil/bin/bash_internal_variables.sh file?

#!/bin/bash
echo -n "! | " ; echo $!

[ snip ]

Post by Philip Rhoades
echo -n "time | " ; echo $time
but why should anything in there matter? Even if an error code was
returned, maildrop should still execute the "to" command?

[ snip ]
I did some tests and now I can confirm your observation. Here is my
explanation.
xfilter - filter message through another program
xfilter expression
expression specifies an external program that maildrop runs
to filter the current message. The current message will be
piped to the filter program as standard input. The output of
the filter program replaces the current message being
delivered. The external program must terminate with an exit
code of 0. If the external program does not terminate with
an exit code of 0, or if it does not read the message from
the standard input, maildrop terminates with an exit code of
EX_TEMPFAIL.
Applied to your example, simplified, line
xfilter "/home/phil/bin/bash_internal_variables.sh > /home/phil/t"
makes output of command "/home/phil/bin/bash_internal_variables.sh

Post by Philip Rhoades
/home/phil/t" new message content. As your redirected stdout to file,

there is no output, which means effectively you deleted the whole
message content. Courier sees that and does nothing, as there is
nothing to deliver.
I think your misused xfilter command. It looks like what you want to do
is some environment logging for testing.

Actually I thought that bash script would just be a convenient test -
what I eventually want to do is run a Ruby script that extracts an CVS
attachment from the mail, processes it and writes a the output to a file
- but of course I do not want to lose the original email . .

Try to use backticks, excerpt
Command substitution
Text enclosed in back-tick characters is interpreted as a shell
command. The shell command is executed as a child process by
maildrop. Its output is used in place of the command. For
DIR=`ls`
places the names of the files in the current directory into the
DIR variable.
The output of the command will have all newline characters
replaced by spaces, and leading and trailing spaces will be
stripped (multiple spaces are not removed, though). Also, the
contents of the message being delivered is made available to the
command on standard input.
Here the message is not overwritten, so line next is executed normally.
So, your test code would be
`/home/phil/bin/bash_internal_variables.sh > /home/phil/t`
instead of
xfilter "/home/phil/bin/bash_internal_variables.sh > /home/phil/t"

That also works but the email is still not delivered as well . . an
example I found in the docs is for reversing characters in lines of
text:

xfilter "$HOME/reverse.sh"

with reverse.sh:

#!/bin/sh

exec 6<&0

while read -u 6 line ;
do
echo $line | rev
done

exec 6<&-
exit 0

(the syntax of which I don't fully understand yet but I am guessing it
is returning the text of the original email?).

I used:

xfilter "/home/phil/bin/reverse.sh > /home/phil/t"

which, like my script worked in creating the appropriate file "t", but
still does not deliver the email for the "to" line . .

Thanks,

Phil.

[Philip Rhoades]

Milan,

On Fri, 05 Feb 2016 08:06:16 +1100
[ snip ]

Post by Philip Rhoades

I think your misused xfilter command. It looks like what you want
to do is some environment logging for testing.

Actually I thought that bash script would just be a convenient test -
what I eventually want to do is run a Ruby script that extracts an
CVS attachment from the mail, processes it and writes a the output to
a file
- but of course I do not want to lose the original email . .

- you receive mail
- you need to extract attachment from it
- you need to save original mail to designated folder
Did I understand it right?

Correct.

[ snip ]

Post by Philip Rhoades
That also works but the email is still not delivered as well . . an
example I found in the docs is for reversing characters in lines of
xfilter "$HOME/reverse.sh"

[ snip ]

Post by Philip Rhoades
xfilter "/home/phil/bin/reverse.sh > /home/phil/t"
which, like my script worked in creating the appropriate file "t",
but still does not deliver the email for the "to" line . .

Because you did the same with other command. Remember, if you are
redirecting stdout (with > /home/phil/t at the end of command), there
is nothing written to stdout, output of command is redirected to file.
The output of the filter program replaces the current message being
delivered.
After xfilter statement is run, whatever is output to stdout is being
delivered instead of original mail message.
In your example, there was no output to stdout (everything was captured
and put into file), so instead of original mail message, empty file was
being considered for delivery. I did not check in sources, but I think
if message is zero sized, no delivery attempt is being made.
To check this, instead of your
xfilter "/home/phil/bin/reverse.sh > /home/phil/t"
use
xfilter "/home/phil/bin/reverse.sh > /home/phil/t; echo any text"
and there should be message for delivery, however, with only 'any
text', not original message as you are trying to do, I think.

You are probably right but I don't see how that helps solve the problem
. . I think it might be easier to take out the xfilter line, let the
mail be delivered as normal and set up an inotifywait script to watch
for new files in the appropriate mail dir and then process them with my
Ruby script . .

Thanks,

Phil.

[Philip Rhoades]

Milan,

On Fri, 05 Feb 2016 09:09:48 +1100

Post by Philip Rhoades
Milan,

[ snip ]

Post by Philip Rhoades

- you receive mail
- you need to extract attachment from it
- you need to save original mail to designated folder
Did I understand it right?

Correct.

Nice, so see below.
[ snip ]

Post by Philip Rhoades
You are probably right but I don't see how that helps solve the
problem . . I think it might be easier to take out the xfilter line,
let the mail be delivered as normal and set up an inotifywait script
to watch for new files in the appropriate mail dir and then process
them with my Ruby script . .

`/home/phil/bin/bash_internal_variables.sh > /home/phil/t`
to $DEFAULT/.tst/

You must have missed my previous comment - while that DID work in terms
of creating the "t" file - the "to" line did NOT work - I just tried it
again - still not working . . this also failed:

xfilter `/usr/bin/date > /home/phil/t`
to $DEFAULT/.tst/

(and replace bash_internal_variables.sh with your real script). That's
all. Believe me, just try it. I did. And I am doing various things with
maildrop. Some trial and error, some man pages reading is all that's
required.

Well I don't know what is wrong with my setup then . .

Now, we can go a bit deeper. If your original mail is correctly
structured according MIME standard, which should be the norm nowadays,
you could just use reformime. In my case, I use in one recipe the
PA=`reformime -e -s 1.1`
If mail MIME structure is the same everytime, there is nothing more to
do, in your case, it could become
`reformime -e -s 1.2 > /home/phil/t`
to $DEFAULT/.tst/
and you will have in file /home/phil/t just the attachment you wanted.
You just need to adjust -s argument according your mail structure.
It is possible to test structure with reformime with no arguments.
For first experiments with reformime, you could just use
reformime < some_file_name_with_saved_mail
to check structure and then add some arguments to do other useful
things.

That is interesting but I have to do some processing on the attachment
before saving the output.

Caveat: I am using full Courier mail suite, so I am not sure reformime
is part of standalone maildrop package you are using, but I think it
is.

Yes, I have it.

Thanks,

Phil.

[Darren Spruell]

Post by Philip Rhoades
Milan,

On Fri, 05 Feb 2016 09:09:48 +1100

Post by Philip Rhoades
Milan,

[ snip ]

Post by Philip Rhoades

- you receive mail
- you need to extract attachment from it
- you need to save original mail to designated folder
Did I understand it right?

Correct.

Nice, so see below.
[ snip ]

Post by Philip Rhoades
You are probably right but I don't see how that helps solve the
problem . . I think it might be easier to take out the xfilter line,
let the mail be delivered as normal and set up an inotifywait script
to watch for new files in the appropriate mail dir and then process
them with my Ruby script . .

`/home/phil/bin/bash_internal_variables.sh > /home/phil/t`
to $DEFAULT/.tst/

You must have missed my previous comment - while that DID work in terms
of creating the "t" file - the "to" line did NOT work - I just tried it
xfilter `/usr/bin/date > /home/phil/t`
to $DEFAULT/.tst/

Milan's example omitted 'xfilter.' No xfilter. Bad xfilter.

- Darren

Post by Philip Rhoades

(and replace bash_internal_variables.sh with your real script). That's
all. Believe me, just try it. I did. And I am doing various things with
maildrop. Some trial and error, some man pages reading is all that's
required.

Well I don't know what is wrong with my setup then . .

Now, we can go a bit deeper. If your original mail is correctly
structured according MIME standard, which should be the norm nowadays,
you could just use reformime. In my case, I use in one recipe the
PA=`reformime -e -s 1.1`
If mail MIME structure is the same everytime, there is nothing more to
do, in your case, it could become
`reformime -e -s 1.2 > /home/phil/t`
to $DEFAULT/.tst/
and you will have in file /home/phil/t just the attachment you wanted.
You just need to adjust -s argument according your mail structure.
It is possible to test structure with reformime with no arguments.
For first experiments with reformime, you could just use
reformime < some_file_name_with_saved_mail
to check structure and then add some arguments to do other useful
things.

That is interesting but I have to do some processing on the attachment
before saving the output.

Caveat: I am using full Courier mail suite, so I am not sure reformime
is part of standalone maildrop package you are using, but I think it
is.

Yes, I have it.
Thanks,
Phil.
--
Philip Rhoades
PO Box 896
Cowra NSW 2794
Australia
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Courier-maildrop mailing list
https://lists.sourceforge.net/lists/listinfo/courier-maildrop

[Bob Bernstein]

Post by Philip Rhoades
if( /^[TC][[co]:.*tst/ )
{
xfilter "/home/phil/bin/bash_internal_variables.sh > /home/phil/t"
to $DEFAULT/.tst/
}

Could someone explain the syntax of the very first line above,
to wit:

if( /^[TC][[co]:.*tst/ )

At first glance I thought I was looking at unbalanced brackets:

[TC][[co]

...but from the discussion I seem to be alone in that opinion.

What is the purpose of this expression? I'm thinking it is
looking to match some combination, with varying cases, of To and
Cc.

Help. please.

:-)

[Philip Rhoades]

Bob,

Post by Bob Bernstein

Post by Philip Rhoades
if( /^[TC][[co]:.*tst/ )
{
xfilter "/home/phil/bin/bash_internal_variables.sh > /home/phil/t"
to $DEFAULT/.tst/
}

Could someone explain the syntax of the very first line above,
if( /^[TC][[co]:.*tst/ )
[TC][[co]
...but from the discussion I seem to be alone in that opinion.
What is the purpose of this expression? I'm thinking it is
looking to match some combination, with varying cases, of To and
Cc.
Help. please.

Actually you have found a typo - which surprisingly didn't prevent the
filter working - I guess it had very infrequent use anyway. One of the
"[["s shouldn't be there and yes, it is a regex that matches either "To"
or "Cc".

Regards,

Phil.

[Dimitri Maziuk]

Post by Philip Rhoades
Actually you have found a typo - which surprisingly didn't prevent the
filter working - I guess it had very infrequent use anyway. One of the
"[["s shouldn't be there and yes, it is a regex that matches either "To"
or "Cc".

I believe in a posixly-correct regex implementation a '[' has no special
meaning inside the range, except for the exceptions. I.e. your regex
would have also matched "T[" and "C[".

[Bob Bernstein]

Post by Philip Rhoades
Actually you have found a typo - which surprisingly didn't
prevent the filter working...

Ah...thanks, and to everyone else who chimed in. An illuminating
discussion!

[e***@s5h.net]

Post by Bob Bernstein

Post by Philip Rhoades
if( /^[TC][[co]:.*tst/ )

Could someone explain the syntax of the very first line above,
if( /^[TC][[co]:.*tst/ )
What is the purpose of this expression? I'm thinking it is looking to
match some combination, with varying cases, of To and Cc.

The brackets had me too. They screamed unclosed to me. However, now
you've asked I ran it through perl:

for i in To Cc T[o C[c ; do echo -e "$i:tst" ; done | perl -ne 'BEGIN
{ use strict; use warnings; } chomp; if( $_ =~ /^[TC][[co]:.*tst/ ) {
print $_, "\n"; }'
To:tst
Cc:tst

Seems legit. Doesn't matter how many [ you place before 'co', same
result every time. Guess perl is just being generous.