Can't make FTPPortmapper work correct, detailed problem definition

Tomasz Jaskolski tj at onyx.pl
Wed Feb 5 20:33:10 EST 2003 

Hello,

I use click router for some time, really like this flexible module
configuration but I'm not able in any of my configurations make
FTPPortMapper work correctly. Either I'm doing something wrong or there is a
bug somewhere.

In order to explain the problem in details I've build the simplest NAT
configuration like this:

// ADDRESS INFORMATION

AddressInfo(
  intern 10.0.0.1                 10.0.0.0/8
00:04:76:15:b0:bf,
  extern 81.210.20.254     81.210.20.0/24         00:04:75:d2:a8:9c,
);

// Interfaces
ToEth0::Queue->IPPrint("InB")->CounterIn::Counter->ToDevice(eth0);
ToEth1::Queue->IPPrint("OutB")->CounterOut::Counter->ToDevice(eth1);
ARPQ0::ARPQuerier(intern);
ARPQ1::ARPQuerier(extern);

IPRewriterPatterns(to_extern_pat extern 1024-65535 - -);

IPRewExtern::IPRewriter(pattern to_extern_pat 0 1, drop);
IPRewExtern[0]->ARPQ1->ToEth1;
IPRewExtern[1]->ARPQ0->ToEth0;

TCPRewExtern::TCPRewriter(pattern to_extern_pat 0 1, drop);
TCPRewExtern[0]->ARPQ1->ToEth1;
TCPRewExtern[1]->ARPQ0->ToEth0;

// eth0 - From inside to outside (intern)
FromDevice(eth0)->
Class0::Classifier(12/0806 20/0001, 12/0806 20/0002, 12/0800);
Class0[0]->ARPResponder(intern)->ToEth0; // ARP Queriers
Class0[1]->[1]ARPQ0; // ARP Responds

// IP packets
IPClass0::IPClassifier(dst net intern, dst tcp port ftp, -);
Class0[2]->Strip(14)->CheckIPHeader->IPPrint("OutA")->IPClass0;
  IPClass0[0]->Discard;  // Discardinternal traffic
  IPClass0[1]->FTPPortMapper(TCPRewExtern, IPRewExtern, to_extern_pat 0 1)->
    TCPRewExtern;  // FTP to world throught NAT
  IPClass0[2]->IPRewExtern; // rest to world throught NAT

// eth1 - from outside to inside (extern)
FromDevice(eth1)->
Class1::Classifier(12/0806 20/0001, 12/0806 20/0002, 12/0800);
Class1[0]->ARPResponder(extern)->ToEth1;  // ARP Queries
Class1[1]->[1]ARPQ1; // ARP Responds

 // IP packets
IPClass1::IPClassifier(dst host extern and src tcp port ftp, dst host
extern);
Class1[2]->Strip(14)->CheckIPHeader->IPPrint("InA")->IPClass1;
  IPClass1[0]->[1]TCPRewExtern; // NAT returning FTP packets
  IPClass1[1]->[1]IPRewExtern; // NAT returning no FTP packets


As you can see IPPrint elements generate some tracking information, where:
A - means before translation
B - after translation
Out - to Outside (direction)
In - to Inside

This configuration works OK. I can make connection from 10.0.0.2 to
81.210.20.252 as www, telnet, etc. and all translations are OK. I incpect
/proc/click/IPRewExtern/* and /proc/click/TCPRewExtern files, the mappings
is being installed when I expect it (even this double mapping done by
FTPPortMapper). The problem begins when I try to use FTP in active mode.
After "ls" command there is no response from the server.

I performed detailed analyse of all traffic and I've noticed some strange
act number behaviour. Take a look at this:

*** first simple www connection ***
Feb  5 20:14:38 dns2 kernel: chatter: OutA: 10.0.0.2.4811 >
81.210.20.252.80: S 1070855980:1070855981(1,60,60) win 32120
*** translation to first free port, which is 1024 ***
Feb  5 20:14:38 dns2 kernel: chatter: OutB: 81.210.20.254.1024 >
81.210.20.252.80: S 1070855980:1070855981(1,74,60) win 32120
Feb  5 20:14:38 dns2 kernel: chatter: InA: 81.210.20.252.80 >
81.210.20.254.1024: S 2474635813:2474635814(1,44,44) ack 1070855981 win
16352
*** responce ack numer (ack 1070855981) is not change through IPRewriter ***
Feb  5 20:14:38 dns2 kernel: chatter: InB: 81.210.20.252.80 > 10.0.0.2.4811:
S 2474635813:2474635814(1,58,44) ack 1070855981 win 16352
Feb  5 20:14:38 dns2 kernel: chatter: OutA: 10.0.0.2.4811 >
81.210.20.252.80: . 1070855981:1070855981(0,40,40) ack 2474635814 win 32120
Feb  5 20:14:38 dns2 kernel: chatter: OutB: 81.210.20.254.1024 >
81.210.20.252.80: . 1070855981:1070855981(0,54,40) ack 2474635814 win 32120
Feb  5 20:14:40 dns2 kernel: chatter: InA: 81.210.20.252.890 >
81.210.20.253.2055: . 3206877270:3206877270(0,40,40) ack 195299455 win 16352
Feb  5 20:14:41 dns2 kernel: chatter: OutA: 10.0.0.2.4811 >
81.210.20.252.80: P 1070855981:1070855988(7,47,47) ack 2474635814 win 32120
Feb  5 20:14:41 dns2 kernel: chatter: OutB: 81.210.20.254.1024 >
81.210.20.252.80: P 1070855981:1070855988(7,61,47) ack 2474635814 win 32120
Feb  5 20:14:41 dns2 kernel: chatter: InA: 81.210.20.252.80 >
81.210.20.254.1024: P 2474635814:2474635859(45,85,85) ack 1070855988 win
16352
Feb  5 20:14:41 dns2 kernel: chatter: InA: 81.210.20.252.80 >
81.210.20.254.1024: F 2474635859:2474635860(1,40,40) ack 1070855988 win
16352
Feb  5 20:14:41 dns2 kernel: chatter: InB: 81.210.20.252.80 > 10.0.0.2.4811:
P 2474635814:2474635859(45,99,85) ack 1070855988 win 16352
Feb  5 20:14:41 dns2 kernel: chatter: InB: 81.210.20.252.80 > 10.0.0.2.4811:
F 2474635859:2474635860(1,54,40) ack 1070855988 win 16352
Feb  5 20:14:41 dns2 kernel: chatter: OutA: 10.0.0.2.4811 >
81.210.20.252.80: . 1070855988:1070855988(0,40,40) ack 2474635859 win 32120
Feb  5 20:14:41 dns2 kernel: chatter: OutA: 10.0.0.2.4811 >
81.210.20.252.80: . 1070855988:1070855988(0,40,40) ack 2474635860 win 32120
Feb  5 20:14:41 dns2 kernel: chatter: OutB: 81.210.20.254.1024 >
81.210.20.252.80: . 1070855988:1070855988(0,54,40) ack 2474635859 win 32120
Feb  5 20:14:41 dns2 kernel: chatter: OutB: 81.210.20.254.1024 >
81.210.20.252.80: . 1070855988:1070855988(0,54,40) ack 2474635860 win 32120
Feb  5 20:14:41 dns2 kernel: chatter: OutA: 10.0.0.2.4811 >
81.210.20.252.80: F 1070855988:1070855989(1,40,40) ack 2474635860 win 32120
Feb  5 20:14:41 dns2 kernel: chatter: OutB: 81.210.20.254.1024 >
81.210.20.252.80: F 1070855988:1070855989(1,54,40) ack 2474635860 win 32120
Feb  5 20:14:41 dns2 kernel: chatter: InA: 81.210.20.252.80 >
81.210.20.254.1024: . 2474635860:2474635860(0,40,40) ack 1070855989 win
16351
Feb  5 20:14:42 dns2 kernel: chatter: InB: 81.210.20.252.80 > 10.0.0.2.4811:
. 2474635860:2474635860(0,54,40) ack 1070855989 win 16351
*** done with port 80, lets try port 21 ***
*** ftp prompt ***
Feb  5 20:14:55 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: S 1088593819:1088593820(1,60,60) win 32120
Feb  5 20:14:55 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: S 1088593819:1088593820(1,74,60) win 32120
Feb  5 20:14:55 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: S 1543749889:1543749890(1,44,44) ack 1088593820 win
16352
Feb  5 20:14:55 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
S 1543749889:1543749890(1,58,44) ack 1088593820 win 16352
Feb  5 20:14:55 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: . 1088593820:1088593820(0,40,40) ack 1543749890 win 32120
Feb  5 20:14:55 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: . 1088593820:1088593820(0,54,40) ack 1543749890 win 32120
Feb  5 20:14:55 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: P 1543749890:1543749962(72,112,112) ack 1088593820 win
16352
Feb  5 20:14:55 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
P 1543749890:1543749962(72,126,112) ack 1088593820 win 16352
Feb  5 20:14:55 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: . 1088593820:1088593820(0,40,40) ack 1543749962 win 32120
Feb  5 20:14:55 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: . 1088593820:1088593820(0,54,40) ack 1543749962 win 32120
*** so far so good, now login name ***
Feb  5 20:14:58 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: P 1088593820:1088593833(13,53,53) ack 1543749962 win 32120
Feb  5 20:14:58 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: P 1088593820:1088593833(13,67,53) ack 1543749962 win 32120
Feb  5 20:14:58 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: . 1543749962:1543749962(0,40,40) ack 1088593833 win
16352
Feb  5 20:14:58 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
. 1543749962:1543749962(0,54,40) ack 1088593833 win 16352
Feb  5 20:14:58 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: P 1543749962:1543749997(35,75,75) ack 1088593833 win
16352
Feb  5 20:14:58 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
P 1543749962:1543749997(35,89,75) ack 1088593833 win 16352
Feb  5 20:14:58 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: . 1088593833:1088593833(0,40,40) ack 1543749997 win 32120
Feb  5 20:14:58 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: . 1088593833:1088593833(0,54,40) ack 1543749997 win 32120
*** password ***
Feb  5 20:15:00 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: P 1088593833:1088593845(12,52,52) ack 1543749997 win 32120
Feb  5 20:15:00 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: P 1088593833:1088593845(12,66,52) ack 1543749997 win 32120
Feb  5 20:15:00 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: . 1543749997:1543749997(0,40,40) ack 1088593845 win
16352
Feb  5 20:15:00 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
. 1543749997:1543749997(0,54,40) ack 1088593845 win 16352
Feb  5 20:15:01 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: P 1543749997:1543750025(28,68,68) ack 1088593845 win
16352
Feb  5 20:15:01 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
P 1543749997:1543750025(28,82,68) ack 1088593845 win 16352
Feb  5 20:15:01 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: P 1088593845:1088593851(6,46,46) ack 1543750025 win 32120
Feb  5 20:15:01 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: P 1088593845:1088593851(6,60,46) ack 1543750025 win 32120
Feb  5 20:15:01 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: P 1543750025:1543750044(19,59,59) ack 1088593851 win
16352
Feb  5 20:15:01 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
P 1543750025:1543750044(19,73,59) ack 1088593851 win 16352
Feb  5 20:15:01 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: . 1088593851:1088593851(0,40,40) ack 1543750044 win 32120
Feb  5 20:15:01 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: . 1088593851:1088593851(0,54,40) ack 1543750044 win 32120
*** ls command in active mode, that means PORT x,x,x,x,x,x is sending and
FTPPortmapper do it's job***
Feb  5 20:15:03 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: P 1088593851:1088593873(22,62,62) ack 1543750044 win 32120
Feb  5 20:15:03 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: P 1088593851:1088593875(24,78,64) ack 1543750044 win 32120
Feb  5 20:15:03 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: P 1543750044:1543750074(30,70,70) ack 1088593875 win
16352
*** now ack number is change from 1088593875 to 3516916288 and that seems to
cause the problems, all further communication fails, the new ack number
appears in packets from server to client every few seconds since now ***
Feb  5 20:15:03 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
P 1543750044:1543750074(30,84,70) ack 3516916288 win 16352
Feb  5 20:15:03 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: P 1543750044:1543750074(30,70,70) ack 1088593875 win
16352
Feb  5 20:15:03 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
P 1543750044:1543750074(30,84,70) ack 3516916288 win 16352
Feb  5 20:15:03 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: P 1088593851:1088593873(22,62,62) ack 1543750044 win 32120
Feb  5 20:15:03 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: P 1088593851:1088593875(24,78,64) ack 1543750044 win 32120
Feb  5 20:15:03 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: . 1543750074:1543750074(0,40,40) ack 1088593875 win
16352
Feb  5 20:15:03 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
. 1543750074:1543750074(0,54,40) ack 3516916288 win 16352
Feb  5 20:15:04 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: P 1543750044:1543750074(30,70,70) ack 1088593875 win
16352
Feb  5 20:15:04 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
P 1543750044:1543750074(30,84,70) ack 3516916288 win 16352
Feb  5 20:15:04 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: P 1088593851:1088593873(22,62,62) ack 1543750044 win 32120
Feb  5 20:15:04 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: P 1088593851:1088593875(24,78,64) ack 1543750044 win 32120
Feb  5 20:15:04 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: . 1543750074:1543750074(0,40,40) ack 1088593875 win
16352
Feb  5 20:15:04 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
. 1543750074:1543750074(0,54,40) ack 3516916288 win 16352
Feb  5 20:15:05 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: P 1543750044:1543750074(30,70,70) ack 1088593875 win
16352
Feb  5 20:15:05 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
P 1543750044:1543750074(30,84,70) ack 3516916288 win 16352
Feb  5 20:15:05 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: P 1088593851:1088593873(22,62,62) ack 1543750044 win 32120
Feb  5 20:15:05 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: P 1088593851:1088593875(24,78,64) ack 1543750044 win 32120
Feb  5 20:15:05 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: . 1543750074:1543750074(0,40,40) ack 1088593875 win
16352
Feb  5 20:15:05 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
. 1543750074:1543750074(0,54,40) ack 3516916288 win 16352
Feb  5 20:15:08 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: P 1543750044:1543750074(30,70,70) ack 1088593875 win
16352
Feb  5 20:15:08 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
P 1543750044:1543750074(30,84,70) ack 3516916288 win 16352
Feb  5 20:15:08 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: P 1088593851:1088593873(22,62,62) ack 1543750044 win 32120
Feb  5 20:15:08 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: P 1088593851:1088593875(24,78,64) ack 1543750044 win 32120
Feb  5 20:15:08 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: . 1543750074:1543750074(0,40,40) ack 1088593875 win
16352
Feb  5 20:15:08 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
. 1543750074:1543750074(0,54,40) ack 3516916288 win 16352
Feb  5 20:15:13 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: P 1543750044:1543750074(30,70,70) ack 1088593875 win
16352
Feb  5 20:15:13 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
P 1543750044:1543750074(30,84,70) ack 3516916288 win 16352
Feb  5 20:15:13 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: P 1088593851:1088593873(22,62,62) ack 1543750044 win 32120
Feb  5 20:15:13 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: P 1088593851:1088593875(24,78,64) ack 1543750044 win 32120
Feb  5 20:15:13 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: . 1543750074:1543750074(0,40,40) ack 1088593875 win
16352
Feb  5 20:15:13 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
. 1543750074:1543750074(0,54,40) ack 3516916288 win 16352
Feb  5 20:15:23 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: P 1543750044:1543750074(30,70,70) ack 1088593875 win
16352
Feb  5 20:15:23 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
P 1543750044:1543750074(30,84,70) ack 3516916288 win 16352
Feb  5 20:15:23 dns2 kernel: chatter: OutA: 10.0.0.2.4812 >
81.210.20.252.21: P 1088593851:1088593873(22,62,62) ack 1543750044 win 32120
Feb  5 20:15:23 dns2 kernel: chatter: OutB: 81.210.20.254.1025 >
81.210.20.252.21: P 1088593851:1088593875(24,78,64) ack 1543750044 win 32120
Feb  5 20:15:23 dns2 kernel: chatter: InA: 81.210.20.252.21 >
81.210.20.254.1025: . 1543750074:1543750074(0,40,40) ack 1088593875 win
16352
Feb  5 20:15:23 dns2 kernel: chatter: InB: 81.210.20.252.21 > 10.0.0.2.4812:
. 1543750074:1543750074(0,54,40) ack 3516916288 win 16352
Feb  5 20:15:42 dns2 kernel: chatter: starting 1 thread
Feb  5 20:15:42 dns2 kernel: click: stopping router thread pid 25119
Feb  5 20:15:42 dns2 kernel: click: starting router thread pid 25121
(ced8b860)

Well gays, what do you think about this?

Tomasz Jaskolski tj at onyx.pl

More information about the click mailing list