Intel igb/em Interfaces Broken on 2.6/22.01+ #67
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: hhf/pfatt#67
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The dhcp lease for connections is not handed through to the ngeth0 interface properly. There isn't any real "errors" in the logs.
If you try to run the script manually after boot you get "ngctl: send msg: File exists"
Logs from pfatt.log:
2022-02-14 14:36:56 :: [pfatt.sh] :: pfSense + AT&T U-verse Residential Gateway for true bridge mode
2022-02-14 14:36:56 :: [pfatt.sh] :: Configuration:
2022-02-14 14:36:56 :: [pfatt.sh] :: ONT_IF: igb0
2022-02-14 14:36:56 :: [pfatt.sh] :: RG_IF: igb1
2022-02-14 14:36:56 :: [pfatt.sh] :: RG_ETHER_ADDR: [MY MAC HERE]
2022-02-14 14:36:56 :: [pfatt.sh] :: attaching interfaces to ng_ether... OK!
2022-02-14 14:36:56 :: [pfatt.sh] :: building netgraph nodes...
2022-02-14 14:36:56 :: [pfatt.sh] :: creating ng_one2many... 2022-02-14 14:37:00 :: [pfatt.sh] :: pfSense + AT&T U-verse Residential Gateway for true bridge mode
I am not running wpa_supplicant mode.
Can confirm its not working after an upgrade. Following the troubleshooting instructions shows that the modules have loaded. PFatt logs dont show anything.
Can confirm it is broken for me as well, running supplicant
Same issue. Not grabbing DHCP.
Edit: I am using the WPA supplicant method.
Want to confirm that reverting to 2.5.2 or 21.05.2 immediately restores internet for me after setting everything back up.
Yes, It was an absolute pain in the a**, but restoring to 21.05.2 immediately fixed it for me. IPv6 wouldn't grab for an hour or so but finally started working.
Also, I posted on the Netgate Forums. If anyone else wants to add anything over there here is the link. https://forum.netgate.com/topic/169882/22-01-2-6-0-upgrade-broke-dhcp-on-wan-interface-with-custom-startup-script
I am having the same problem and now my WireGuard and other tools don't work and can't get them to work.
Yep - supplicant not working for me either. The last time a new version of pfsense broke pfatt Matt Johnson submitted this issue to pfsense redmine. Should we do that here? Here is the issue that originated the whole thing.
It also broke mine after update. Per the docs, I ran "tcpdump -ei ONT_IF" and "tcpdump -ei RG_IF", which should filter and capture link layer information (2), on my interfaces and captured 0 packets from RG_IP and only the bridged DHCP traffic on the ONT_IF interface.
I reset netgraph, which removes the hooks, rebooted the gateway and modem with tcpdump running and captured 0 packets from the interfaces. Before removing the netgraph hooks, the only traffic I seen on any of the three interfaces, was the DHCP request on the ngeth0 virtual interaface, and the bridged ONT_IF interface. So the DHCP requests are still getting to the correct interface.
The fact that tcpdump doesn't see any traffic makes me think its being filtered, like promisc mode isn't allowing EAPOL 802.1X traffic to be capture, and there fore is not bridged. No authentication mean no DHCP response. IMO
I've moved to inline behind the gateway until this can be figured out. I would be willing to test once a day.
Okay, had some success today based on info I gathered from all the various discussions online. I think it is something to do with the em(4) driver. Do all of you having issues have Intel NIC's? I put together a test pfSense server from a bunch of spare parts and it worked right away on the latest release. After digging, I couldn't get any Intel NIC to work. Using what I had around (a few crappy USB dongles worked and old PC's with integrated NICs) I had success with everything not Intel GbE. When I re-upgraded my main pfSense box I was able to move my WAN link to an SFP slot (with RJ45 Module) with some success. I say "some" because all my SFP/RJ45 modules are 10GB and they do not negotiate well with the ONT.
Something interesting for me, if_em.ko is present in /boot/kernel on 2.6.0 but wasn't in my previous version of pfSense. My knowledge is limited but I am not sure where the driver was located in the previous version? Anyone smarter than me know?
Some Useful Links:
FreeBSD 12.3 Release Notes (em(4) driver notes) - https://www.freebsd.org/releases/12.3R/relnotes/
Reddit Discussion - https://www.reddit.com/r/PFSENSE/comments/ssgsha/psa_260_breaks_att_bypass/?sort=new
Netgate Forum Discussion - https://forum.netgate.com/topic/99190/att-uverse-rg-bypass-0-2-btc/396?_=1644931323812
OPNSense GIT Issue - https://github.com/MonkWho/pfatt/issues/65
I think this is going somewhere because I've tried multiple different boxes but they're all Intel Nics, when I get off work I will try a couple USB dongle's to see if it gets traffic that way.
The USBs work for me but are slow. Download is like 100m, upload is better at around 400m. I have a 1G SFP that should get here tomorrow. Really hoping that talks better with the ONT then the 10G did.
For USBs to work at 1 gig speeds you have to have 3.1 USB port or better. For FreeBSD, I am using a box equivalent to the netgate 1541 Same everything but a lot more powerful. Let me know how it goes with the other Nics.
Will do! If it helps I'm using the XG-1537 so USB3.0
Is the usb dongles 3.0, when I was using usb in past it worked great I was able to get full 1gb speeds out of my usb ports. If the usb is 3.0 then I don't know why I am getting full 1gb speeds. But I did downgrade back to 2.5.2 now WireGuard don't work on 2.5.2.
Yep!
Nothing useful to add here but I can confirm I'm using an Intel NIC with the em driver. Neither tethered or supplicant working for me on 22.1 but supplicant is working on 21.7.8
em0: <Intel(R) 82583V> port 0xe000-0xe01f mem 0xdf500000-0xdf51ffff,0xdf520000-0xdf523fff irq 16 at device 0.0 on pci1
em1: <Intel(R) 82583V> port 0xd000-0xd01f mem 0xdf400000-0xdf41ffff,0xdf420000-0xdf423fff irq 17 at device 0.0 on pci2
em2: <Intel(R) 82583V> port 0xc000-0xc01f mem 0xdf300000-0xdf31ffff,0xdf320000-0xdf323fff irq 18 at device 0.0 on pci3
em3: <Intel(R) 82583V> port 0xb000-0xb01f mem 0xdf200000-0xdf21ffff,0xdf220000-0xdf223fff irq 19 at device 0.0 on pci4
em4: <Intel(R) 82583V> port 0xa000-0xa01f mem 0xdf100000-0xdf11ffff,0xdf120000-0xdf123fff irq 16 at device 0.0 on pci5
em5: <Intel(R) 82583V> port 0x9000-0x901f mem 0xdf000000-0xdf01ffff,0xdf020000-0xdf023fff irq 17 at device 0.0 on pci6
I'm on a Protectli FW6D
I am using an Intel NIC but with the IGB driver.
And is it working or not because my system is using igb drivers too and mine is not working
My knowledge on FreeBSD is limited but I believe igb uses the em(4) driver. All the common Intel cards fall under it (I350, 82575, etc.)
https://www.freebsd.org/releases/12.3R/hardware/
Not working
If you look at the if_igb.ko driver in /boot/kernel it just is a shortcut to if_em.ko. I think at one point the two intel drivers merged. https://www.intel.com/content/www/us/en/download/15187/intel-network-adapter-gigabit-base-driver-for-freebsd.html?wapkw=i350%20freebsd
Okay, I got everything up and working on my regular Intel NIC. I’m not the biggest expert here so bear with me.
Through troubleshooting I was able to get every non-Intel NIC to authenticate and pull DHCP. After more testing all igb(4) driver-based cards failed. In the /boot/kernel folder I noticed if_igb.ko is simply a shortcut to the em(4) driver (if_em.ko). I am guessing FreeBSD is using this combined driver from intel? https://www.intel.com/content/www/us/en/download/15187/intel-network-adapter-gigabit-base-driver-for-freebsd.html
Alternatively, I found this driver that appears to be for igb(4) separately, and it seems newer. https://www.intel.com/content/www/us/en/download/14610/intel-network-adapter-driver-for-82575-6-and-82580-based-gigabit-network-connections-under-freebsd.html?wapkw=i350%20freebsd
I downloaded a FreeBSD-12.3 VM, its related source code (amd64), and complied the separate igb(4) driver.
I loaded my newly compiled if_igb.ko into the /boot/modules folder with chmod 555 permissions. Next, I added the following two lines to my /boot/loader.conf file to supersede the included driver.
if_igb_load="YES"
if_igb_name="/boot/modules/if_igb.ko"
Rebooted and everything came up just fine!
Feel free to use my compiled if_igb.ko if you don’t want to build your own.
https://github.com/neydah700/pfsense_intel/blob/main/if_igb.ko
Also, for reference here is my pfatt script if anyone needs a reference.
https://github.com/neydah700/pfsense_intel/blob/main/pfatt_intel.sh
A few notes:
Interesting that the intel igb driver works. I searched for bugs on the FreeBSD buglist and found this...
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260068
Looks like it might be related? Issues with vlan tagging. Was introduced in 13.0 and 12.3... recently fixed in the stable branches, so the timing lines up.
Some comments and feedback in testing so far:
It seems safe to install and test this on 2.5.2. I have downloaded the kernel module and am testing prior to any updates. I haven't managed to break 2.5.2... yet.
It would be better to create /boot/loader.conf.local instead of /boot/loader.conf. Loader.conf may be overwritten by pfsense updates.
What is your output on 2.6.0 with the if_igb.ko module for "kldstat -v"? I can't confirm it is loading and in use on 2.5.2. I am reluctant to upgrade until I can validate it is loading.
Could explain why we are passing 802.1x not pulling DHCP on VLAN 0. I'll add it to my redmine issue on pfSense. If anyone else has success can they go on and comment. Hopefully we get some traction! https://redmine.pfsense.org/issues/12821?next_issue_id=12820
I am testing now reimaging since wiregraud is broke in my install right now.
i am testing now reimaging since wiregraud is broke in my install right now.
will do internet going out for a bit to update and bring system online.
Good point on the .local, will adjust that.
For my kldstat does just this portion work for ya or do you want the whole output?
3 1 0xffffffff83cfb000 35e08 if_igb.ko (/boot/modules/if_igb.ko)
Contains modules:
Id Name
2 pci/igb
Thank you. That is what I was curious about. It isn't loading on 2.5.2, but it may just be because it was compiled for a different kernel. I also can't get into load with kldload on 2.5.2.
I built it using 12.3-RELEASE on amd64 architecture. Hopefully that helps!
Time to cross fingers and see how this goes....
Ok, some lessons learned.
pfSense doesn't have wget or curl packages installed by default I didn't want to start mucking up the default packages. It does have fetch. However for whatever reason, when I used fetch to grab the module it was corrupted. After downloading the module on a desktop and uploading it to the firewall with Cyberduck, this worked perfectly. It might work in 2.5.2, but now I have no way of testing. I am now going to try to upgrade to pfSense Plus. I will report. Thank you!
Upgrade to 22.01 from 2.6.0 was perfectly smooth with this workaround in place.
My recommendation to anyone on CE 2.5.2 is to download the module on a desktop and then upload it to your firewall since I encountered issues fetching directly to pfSense. Set up your loader.conf.local as described above. The upgrade to 2.6.0 should then proceed without issue and eventually come back up with a successfully bypassed AT&T gateway. Based on the fix being related to the Intel kernel driver not properly tagging VLAN through netmap, I think this will resolve issues for tethered or supplicant methods. If you wish to register and upgrade to Plus, you can also now do that.
@jasonsansone Glad you are back up! If you get a chance do you mind adding a comment over on the Redmine issue? Hopefully we can get this fixed natively! https://redmine.pfsense.org/issues/12821?next_issue_id=12820
Does everyone who this has worked for extract their own certs?
I noticed you are using the EAP_Identity to apply to the MAC address on ngeth0.
Before when this was working for me I would have to apply the RG MAC to that but then use the MAC that came with the certs in the EAP_Identity part.
Ah good point. My EAP and RG MAC are the same so I simplified my script. If someone uses different ones they may need to tailor my script a bit.
I never even looked at the new script. I’m using the same I always have. The newly compiled module was all I needed.
@jasonsansone would you mind posting your script, I cant seem to get mine working with my probably incorrect additions
I’m using the tether method. I never bothered to mess with extracting certificates when I set this up years ago and having the gateway sit there didn’t bother me.
The last update broke wpa_supplicant (luckily fixed) and this release broke the VLAN 0 piece. I am strongly considering going back to the tether method to reduce customizations.
Interesting, the tether worked but only after i did the interface assignment from the console, using the assignments within the UI didn't help and then I had to reboot.
Now that ngeth0 is assigned I am going to try and flip the script back to the intel script and see if it works.
@bigjohns97 if you want, I just made a commit to my script to break out the RG and EAP MAC addresses. I haven't tested it but it should work.
https://github.com/neydah700/pfsense_intel/blob/main/pfatt_intel.sh
@neydah700 Thanks for figuring out the fix. I posted to my website to hopefully help people find the fix faster. Please let me know if you think anything should be added or removed to it.
https://angrysysadmins.tech/index.php/2022/02/grassyloki/pfsense-2-6-0-fix-pf-att-bypass-mode/
Thanks for sharing! One note, while this seems to be fixing the ibg(4) people I am guessing it is not fixing the em(4) people. There is more discussion going on over at the Netgate Forums. I am going to compile the combined driver and see if that still works for me. So maybe will need an update if this works!
Thanks for everything, unfortunately this didn't work for me but it did tell me my script was correct, the only difference between yours and mine was mine had an extra space on the logger line.
I guess I am stuck running tether mode from now on until someone maybe can solve this issue I had.
Maybe I go and try to extract the actual certs from my device at a later date and see if it's the certs that are bad or if they aren't allowing the interface MAC to not match the EAP_Identity MAC now or something.
@bigjohns97
Sorry about that! for what it's worth I have never used anything but the same MAC's, so my script might be wrong. I guess only other suggestion is make sure pfSense is not managing that interface and that you set the ngeth0 MAC address to match your RG MAC in the web interface?
I tried adding this but it didn't make a difference, tether was working without it as well so I am hanging it up for the day until I get get some time to get those certs out of the RG.
Thanks again @neydah700
i got the certs out again and now it don't work I cant get pass the
WPA_STATUS_CMD="wpa_cli status | grep 'suppPortStatus' | cut -d= -f2"
IP_STATUS_CMD="ifconfig ngeth0 | grep 'inet\ ' | cut -d' ' -f2"
/usr/bin/logger -st "pfatt" "waiting for EAP authorization..."
During all this messing. I deleted my cert, so I had to pull again.
=Here is my full script
#!/usr/bin/env sh
EAP_SUPPLICANT_IDENTITY=""
RG_ETHER_ADDR=""
LOG=/var/log/pfatt.log
ONT_IF="igb0"
getTimestamp(){
echo
date "+%Y-%m-%d %H:%M:%S :: [pfatt.sh] ::"}
DO NOT EDIT BELOW
/usr/bin/logger -st "pfatt" "starting pfatt..."
/usr/bin/logger -st "pfatt" "configuration:"
/usr/bin/logger -st "pfatt" " ONT_IF = $ONT_IF"
/usr/bin/logger -st "pfatt" " EAP_SUPPLICANT_IDENTITY = $EAP_SUPPLICANT_IDENTITY"
/usr/bin/logger -st "pfatt" " RG_ETHER_ADDR = $RG_ETHER_ADDR"
Netgraph cleanup.
/usr/bin/logger -st "pfatt" "resetting netgraph..."
/usr/sbin/ngctl shutdown $ONT_IF: >/dev/null 2>&1
/usr/sbin/ngctl shutdown vlan0: >/dev/null 2>&1
/usr/sbin/ngctl shutdown ngeth0: >/dev/null 2>&1
/usr/bin/logger -st "pfatt" "your ONT should be connected to pyshical interface $ONT_IF"
/usr/bin/logger -st "pfatt" "creating vlan node and ngeth0 interface..."
/usr/sbin/ngctl mkpeer $ONT_IF: vlan lower downstream
/usr/sbin/ngctl name $ONT_IF:lower vlan0
/usr/sbin/ngctl mkpeer vlan0: eiface vlan0 ether
/usr/sbin/ngctl msg vlan0: 'addfilter { vlan=0 hook="vlan0" }'
/usr/sbin/ngctl msg ngeth0: set $RG_ETHER_ADDR
/usr/bin/logger -st "pfatt" "enabling promisc for $ONT_IF..."
/sbin/ifconfig $ONT_IF ether $RG_ETHER_ADDR
/sbin/ifconfig $ONT_IF up
/sbin/ifconfig $ONT_IF promisc
/usr/bin/logger -st "pfatt" "starting wpa_supplicant..."
WPA_PARAMS="
set eapol_version 2,
set fast_reauth 1,
ap_scan 0,
add_network,
set_network 0 ca_cert \"/root/pfatt/wpa/ca.pem\",
set_network 0 client_cert \"/root/pfatt/wpa/client.pem\",
set_network 0 eap TLS,
set_network 0 eapol_flags 0,
set_network 0 identity \"$EAP_SUPPLICANT_IDENTITY\",
set_network 0 key_mgmt IEEE8021X,
set_network 0 phase1 \"allow_canned_success=1\",
set_network 0 private_key \"/root/pfatt/wpa/private.pem\",
enable_network 0
"
WPA_DAEMON_CMD="/usr/sbin/wpa_supplicant -Dwired -i$ONT_IF -B -C /var/run/wpa_supplicant"
Kill any existing wpa_supplicant process.
PID=$(pgrep -f "wpa_supplicant")
if [ ${PID} > 0 ];
then
/usr/bin/logger -st "pfatt" "terminating existing wpa_supplicant on PID ${PID}..."
RES=$(kill ${PID})
fi
Start wpa_supplicant daemon.
RES=$(${WPA_DAEMON_CMD})
PID=$(pgrep -f "wpa_supplicant")
/usr/bin/logger -st "pfatt" "wpa_supplicant running on PID ${PID}..."
Set WPA configuration parameters.
/usr/bin/logger -st "pfatt" "setting wpa_supplicant network configuration..."
IFS=","
for STR in ${WPA_PARAMS};
do
STR="$(echo -e "${STR}" | sed -e 's/^:space:*//')"
RES=$(eval wpa_cli ${STR})
done
Create variables to check authentication status.
WPA_STATUS_CMD="wpa_cli status | grep 'suppPortStatus' | cut -d= -f2"
IP_STATUS_CMD="ifconfig ngeth0 | grep 'inet\ ' | cut -d' ' -f2"
/usr/bin/logger -st "pfatt" "waiting for EAP authorization..."
Check authentication once per 5 seconds for 25 seconds (5 attempts). Continue without authentication if necessary (no WAN).
i=1
until [ "$i" -eq "5" ]
do
sleep 5
WPA_STATUS=$(eval ${WPA_STATUS_CMD})
if [ X${WPA_STATUS} = X"Authorized" ];
then
/usr/bin/logger -st "pfatt" "EAP authorization completed..."
done
@SGC1990 what folder are your certs in and do they have the appropriate permissions? That script assumes they are in the /root/pfatt/wpa/ directory.
also, are you using igb0?
and the ont is plug in to igb0
This is what happens to mine as well, unable to EAP auth.
also, the script is not making a log any more
do you have teather working
Yes tether works for me, do you have your own certs or someone else's?
I have my own.
@SGC1990 yea the logging broke for me as well so I ended up removing it from my script. No idea why the log stopped but that wasn't my pressing issue :).
Idk what could be different for you, if tethering works and we are using the same script. If it's not passing EAP maybe the certs need a format conversion or something? Maybe try changing the permissions on the certs themselves to something like 755 and see if it makes a difference?
tething is not working since i lost the script for that ether.
@SGC1990 .. @jasonsansone is using tethering, maybe he can share his script with you?
Here is the tether script I used
https://github.com/MonkWho/pfatt/blob/master/bin/pfatt.sh
It's just the one from this repo.
Thank you that worked. I am up but I still want to get rid of the gateway it is taking up room.
Can anyone help me figure out why the logs don't work any more.
`#!/bin/sh
set -e
ONT_IF='igb0' # CHANGE THIS VALUE
RG_IF='em0' # CHANGE THIS VALUE
RG_ETHER_ADDR='00:00:00:00:00:00' # CHANGE THIS VALUE
LOG=/var/log/pfatt.log
{
/usr/local/bin/php -r "pfSense_ngctl_attach('.', '$ONT_IF');"
/usr/local/bin/php -r "pfSense_ngctl_attach('.', '$RG_IF');"
/usr/sbin/ngctl mkpeer $ONT_IF: one2many lower one
/usr/sbin/ngctl name $ONT_IF:lower o2m
/usr/sbin/ngctl mkpeer o2m: vlan many0 downstream
/usr/sbin/ngctl name o2m:many0 vlan0
/usr/sbin/ngctl mkpeer vlan0: eiface vlan0 ether
/usr/sbin/ngctl msg vlan0: 'addfilter { vlan=0 hook="vlan0" }'
/usr/sbin/ngctl msg ngeth0: set $RG_ETHER_ADDR
/usr/sbin/ngctl mkpeer o2m: etf many1 downstream
/usr/sbin/ngctl name o2m:many1 waneapfilter
/usr/sbin/ngctl connect waneapfilter: $ONT_IF: nomatch upper
/usr/sbin/ngctl mkpeer $RG_IF: etf lower downstream
/usr/sbin/ngctl name $RG_IF:lower laneapfilter
/usr/sbin/ngctl connect laneapfilter: $RG_IF: nomatch upper
/usr/sbin/ngctl connect waneapfilter: laneapfilter: eapout eapout
/usr/sbin/ngctl msg waneapfilter: 'setfilter { matchhook="eapout" ethertype=0x888e }'
/usr/sbin/ngctl msg laneapfilter: 'setfilter { matchhook="eapout" ethertype=0x888e }'
/usr/sbin/ngctl msg o2m: setconfig "{ xmitAlg=2 failAlg=1 enabledLinks=[ 1 1 ] }"
/usr/sbin/ngctl rmhook waneapfilter: nomatch
/sbin/ifconfig $RG_IF up
/sbin/ifconfig $ONT_IF up
/sbin/ifconfig $RG_IF promisc
/sbin/ifconfig $ONT_IF promisc
} >> $LOG`
I can't directly answer that question, but dmesg can still be used to debug most of the creation stage of the interfaces and net graph flow.
I'm tethered and not having much success. I have applied the patches but I'm not getting a DHCP on ngeth0.
Any ideas?
Hey! It looks like your NICs are using the em module and not igb. Unfortunately my igb module doesn't help with em devices. I have tried compiling an em module but didn't have a chance to test. Just uploaded here, https://github.com/neydah700/pfsense_intel/blob/main/if_em.ko
send us the code I had to change the code to get tethering working.
@SGC1990 the code? you mean the script? I'm using the pre-canned one here.
https://github.com/MonkWho/pfatt/blob/master/bin/pfatt.sh
@neydah700 thanks for trying. Same result with the em module. What I did is below.
@bbrendon Meh. I was worried that module wasn't going to work. It is supposed to be a combined igb/em driver but it never loaded properly for me.
If your willing can you run the command below with the module loaded?
sysctl dev.em.0It should spit out a lot of lines. Do you see any that start with dev.em.0.iflib? If not I think the driver loaded properly and still has the issur. If you do see them I think my em driver isn't working. If you don't see any iflib lines let me know and I'll try compiling an older driver version.
@neydah700 SUCCESS! Reviewed it again by reading man pages for loader.conf and I realized I deleted _load from the em line in the conf file. Fat fingers.
Anyway. Fixed the typo and all good.
Thanks for the help and the if_em.ko file!
EDIT. I think dmesg shows em0 differently now. It shows the below which I think is different than previously?
@bbrendon glad it's working! Did you end up using the if_igb.ko file or the if_em.ko file in the final working version?
Edit: I didn't read your full post, 7.7.8 is the version of the em driver I compiled so it looks like it took!
Just want to say thank you to @neydah700 I managed to upgrade mine to 2.6. My system has both em (Intel I219-LM) and igb (Intel 82756). I decided to use both driver compiled by @neydah700.
I use supplicant mode and do fresh 2.6 installation instead of upgrade from 2.5.2
Driver compiled by neydah700:
https://github.com/neydah700/pfsense_intel/raw/main/if_em.ko
https://github.com/neydah700/pfsense_intel/raw/main/if_igb.ko
content of my loader.conf.local
Steps:
If you check dmesg, em driver should be version 7.7.8 and igb should be 2.5.21
So you have it work without gateway pluged in with certs download from the gateway.
That the solution with the igb drivers + supplicant method works for me. Thank you @neydah700
@yovio, @grevelle Could you please post the script you are using?
I submitted a pull request a while back with an updated procedure for enabling supplicant mode to bypass the AT&T gateway. Here is the script.
I'm using this script: https://github.com/MonkWho/pfatt/blob/supplicant/bin/pfatt.sh
It's in supplicant branch
Yes, people call it wpa_supplicant method.
I have gotten wap_supplicant and tethering working. I will post an updated way I have tested on different boxes with intel cards and Onces without and it works with doing the update if you change over to the way I post. I am going to pull all the info together there are lots of different info.
In case this might help others. Before my successful attempt above. I did try using upgrade process instead of clean install of 2.6. In that process, I managed to get WAN IP using wpa_supplicant, so unlike most report that failed to get DHCP, I was able to, but for some reason the DNS resolver is not working fine, I did remember try using DNS Forwarder, its slightly better but connection seems really bad. The latest attempt by doing clean install is perfect.
What chipset are your NICs using? Intel NICs seemed to be the only ones effected. In particular igb and em driver cards.
So far, I have uploaded all the files need to do the wpa_supplicant and tethering. I will have the setup info in the next few days.
This works on all systems ranging from intel and beond since the info did not help me that much, I learned how to do it without this info. This is more work. But I have a step-by-step setup, I have tested this on 33 devices so far and they came up first time. Here is the link https://github.com/SGC1990/pfatt
Thanks again neydah700 and bigjohns97
Show a picture
Supplicant info is a work in progress. but it should be ready for anybody to use. I am here to help if anyone needs it.
I fear that I am not going to be able to go back to supplicant mode until I extract the certs from my actual RG.
I was using purchased certs which don't match the rg gateway Mac address.
This was never a problem before but seems to be a problem now for whatever reason.
the docs from the supplicant branch has the info to pull the certs from the rg. Takes about 10 mins.
I'll check it out, thanks.
you do not even have to take the internet down just connect to wifi on the rg and then follow the docs just make sure to download Python3 for windows first.
I am not familiar with this method, can you link me to the doc?
The only one I know about is from devicelocksmith which requires root to the device or a soldiering gun.
Here is the link https://github.com/SGC1990/pfatt/tree/supplicant
No Soldiering required.
Looks like it won't take 1.0.29 due to "incompatibility"
Is this what 2.7.7 is for?
Do I load 2.7.7 and then load 1.0.29?
just load 1.0.29 and leave it, 2.7.7 is if you wont to go back.
hold it witch modle do you have
BGW210-700
what verison is on the device now..
2.16.4
try to down grade to 2.7.7
Same message
Firmware Upgrade failed due to version incompatibility.
https://www.reddit.com/r/ATT/comments/ouzll8/downgrade_bgw210700_on_firmware_2134_for_wpa/
Looks like this is when it started, no more custom firmwares.
I wiill look into it and get back to you.
Great thanks this seems to work. I am on the bypass method, not the WPA one and this seemed to have done the trick as far as I know. I rebooted pfsense and it still wouldn't work, but after rebooting my BGW210(while pfsense was running) then everything started to work.
So I do not know if I actually needed the driver technically, but I am sure I did.
I am using an em Intel adapter and it loads just fine
Now to remember that I installed this driver a few years from now ;)
Confirmed that the -vlanhwtag -vlanhwfilter -vlanhwtso on ifconfig worked on both 22.1 and 22.1.2_1 for me.
I can't seem to upgrade my NVG589 to the firmware in the linked repository.
Seems like I get firmware incompatibility issues.
They have blocked this by not allowing custom firmware.
So then what do we do? I have to stick to the bridge mode then yes? I have the Pace currently.
I have had to stick to bridge mode, wish I would have known about how easy it was to extract certs before ATT blocked this option.
Wow, that is unfortunate. Would have saved me some money on ebay. Good thing this worked for me as well:
Hopefully I wont keep having regular downtimes anymore.
That worked on pfsense?
Sorry, this was on opnsense, latest version with an intel NIC.
has anyone had this issue with realtek devices (re0/re1)? its not using the if_em drivers.
Can we confirm you can't use the supplicant since you can't revert old att boxes to specific firmware?
I bought a NVG589 and can't update the firmware. There seems to be a lot
of incorrect information in many places for the supplicant setup.
Also, I have not seen this anywhere. Where have you heard this? I am confused as I have never seen this in any of the issues here.
https://www.reddit.com/r/ATT/comments/ouzll8/downgrade_bgw210700_on_firmware_2134_for_wpa/
I also tried applying the firmware posted to my own device and found the error message that led me to the above reddit thread.
Is there a reason why this is not mentioned anywhere?
Is this because of secrecy or something? If so thats fine. I have talked to ATT tech support mentioning their horrific service, and will be getting a BGW210-700.
Bridge mode for the PACE is a nightmare so maybe bridge mode for the BGW will be better I guess.
Just to add here, I have not mentioned anything in regards to this. I just told them that the PACE is shit.
Careful, you're likely to get the BGW320 and not be able to bypass at all.
Then I will probably have to move to another internet company then. I cannot deal with intermittent downtimes while working from home. This just started about 2 months ago, and I am not sure why.
Regardless, if they send me the wrong equipment I will just have to return it back and cancel my sub.
I am good now. I was able to extract on a lower firmware version. Thanks for the help all.
Please do share, how did you get to the lower firmware version?
I am stuck on tether and would love to unplug the RG and throw it back in the closet.
Hey,
Just an FYI I am using opnsense. I went through this tutorial and realized the ping command here: https://github.com/MakiseKurisu/NVG589/wiki/Root-Access, was wrong now.
It should be
`ping telnetd -l sh -p 9999 192.168.1.254`(Or whatever your gateway ip address is)
Once I got it to work I was able to run the rest of the process.
telnet 192.168.1.254 9999I was able to downgrade to this firmware to be able to get this done: https://github.com/MakiseKurisu/NVG589/blob/master/nbxvu9.1.0h4d38.bin
This allowed me to do the above.
Ah I see you have the NVG589
Yeah, I still have an order for the BGW210 but for now im good.
I probably made some dumb rookie moves, but I figured it out in the end. You live, and you learn.
Been digging into this a lot and there's quite a bit to digest.
Has anybody tested on 2.7 or 22.05 DEVEL branches of pfSense at this point? There was a bunch of patches including the FreeBSD 12.3 em driver patches, I think, in the latest kernel builds.
When I tried this on pfSense 2.6, the startup sequence was outputting to the console at about one character per second. I couldn't bear the downtime to wait for it to finish. Any idea why it would move so slow? Also, I had to manually select VGA console from the bootup menu to see any output at all, whereas normally this just happens by default.
I might be willing to risk this (running bare metal) could you point me to the reason why you suggest that the drivers were updated?
More pointedly, has anyone been using this solution with pfSense set to use a RAM disk? Should this make a difference?
I sort of got it working by following this post. Unfortunately, there's a lot of custom pfSense configuration that this broke, and I couldn't manage to sort it out before deciding I didn't want to incur any more downtime trying to figure it out. I realize this is beyond the scope of this particular issue, but if anyone could give me some pointers, I'd appreciate it.
My standard config is thus:
While I could get it to communicate with the rest of the internet by changing the WAN interface to DHCP, the IP address it was obtaining wasn't my static IP, and none of my OpenVPN connections would complete. At no point was the Broadband light on the BGW210 solid green.
Anyhow, if anyone else is using this with a static IP, can you please share what additional configuration you had to do to get it working?
I have a block of 32 ips. All you have to do to get the ip working is to set the ip of ngeth0 to dhcp and then setup up you ip under Virtual IPs, then go to nat and set nat to Hybrid Outbound NAT rule generation. and make a nat to the dedicated ip now the lan will be using the dedicated ip. If you have more then one then you can keep adding then to Virtual IPs, and keep using nat to send then out back local client.
If you need anymore help let me know I have mine setup with open port to dedicated ips and more.
i simply left the public IP's as nat's in my setup. i have ngeth0 set to dhcp, and use NAT's to handle my /29 block. It allows me to utilize all 8 IP's after a fashion (the first and last won't allow for inbound traffic, but outbound works without issue, like for IOT networks). All other 6 IP's use both internal and external traffic based on the NAT table Firewall => NAT => etc.
All you have to do is use 1:1 nat and then go to the firewall and map it.
1:1 NAT isn't an option for me, because I have two OpenVPN servers (site-to-site and remote) running on pfSense under my single public IP.
Virtual IP sounds like it could work, but if I understand correctly, it would require changing every Inbound NAT, Outbound NAT (I use Manual), VPN Client/Server, and Firewall Rule that references my WAN address (whether as the interface or destination) to instead reference the Virtual IP.
do you have the openvpn running in pfsense
Yes.
then just use Virtual IP then make a nat rule that is 127.0.0.1 to that ip and then open firewall ports you should be good to go
I use 1:1 and have 1 address dedicated to openvpn. Having a second is just as easy, and you don't even need to worry about the virtual ip shenanigans.you shouldn't need to even reconfigure your rules either
Somthimes on reboot it does something with the script and makes the system go unresponsive and then I had to remove all my configurations and then we add them back again.
I did end up using Virtual IP, and I did have to replace a lot of references to WAN address in my config, but it was pretty straightforward. The first time I tried, I had failed to set the MAC address on the WAN interface to the BGW210 MAC. Another thing I had to was change the default gateway from StaticIP+1 to StaticIP. The only major snag I've hit in updating my rulesets is one that forwarded traffic bound for the LAN IP of the BGW210 out the WAN interface. Because the BGW210 is now only connected on the ONT port instead of one of the LAN ports, this doesn't work. I do have a spare interface on the firewall that I could connect to one of the LAN ports for this purpose. I know I don't want to enable WiFi on it. Curious how others handle access to the web interface on the BGW210 in tethered mode.
I've been too busy to formally submit any bugs. The FreeBSD upstream fixes are all over the place, I'm with you, hard to follow.
I re-enabled vlanhwtso and my setup still works fine. Confirming your experience.
For reference, I have the vlanhwtag vlanhwfilter enabled on my LAN em0 Intel interface with multiple VLANS in OPNSense and it works fine. I do think it is an igb related FreeBSD driver bug that these HW vlan offloading options don't work.
Additionally it looks like the 22.1.4 codebase for the e1000 driver (igb and em in Freebsd) remains unchanged with a Dec 22, 2021 modified date. So, these disabling options are still needed for the time being.
https://github.com/opnsense/src/tree/22.1.4/sys/dev/e1000
Tested on pfSense Plus 22.05 and getting the following:
Fatal error: Uncaught Error: Call to undefined function pfSense_ngctl_attach() in Command line code:1
Stack trace:
#0 {main}
thrown in Command line code on line 1
I also tried running the PHP code directly without the variables and it threw the same errors to console.
I'll attach this to the redmine as well.
The pfSense_ngctl_attach() and pfSense_ngctl_detach() php modules are being removed in pfSense Plus 22.05 and pfSense CE 2.7, which is why this command is failing on the latest builds. These commands are apparently no longer needed as all interfaces will always be a part of netgraph in the newer release, so these lines should be able to be commented out. I'll test with these lines removed in the next few days. If it works without them, I'll create a fork and/or open an issue with the project to correct it.
A bit different than the Intel issue being discussed previously but that one's actually easy to fix. Added a pull that supports old and newer versions of pfSense while people update.
Currently running opnsense so I haven't tested it past confirming it parses correctly but its pretty straight forward. If you want to give it a shot it'd be appreciated.
Thank you for the patch neclimdul! I've tested with the patched script. My testing can be found here:
https://redmine.pfsense.org/issues/12821#note-14
Seems that on internal builds of pfSense Plus 22.05 there is still an issue with DHCP over VLAN0 on the included Intel driver. I've raised the flag on this internally so hopefully there will be a fix soon. Even if it isn't, though, things should keep humming along for non-Intel NICs or with the custom driver, which I'll compile a new one of and post here from my build environment.
FYI to anyone not watching the redmine or who stumbles on this bug report, the issue ONLY affects igb/em interfaces. Intel ix or igc are not affected and likely Realtek, Broadcom, or any other NIC should be fine as well. For igb/em interfaces, just compile a driver from the Intel provided driver above and load in the driver with the loader.conf.local override.
@ChronicledMonocle would you then suggest those of us with Realtek, Broadcom etc chippers create a different bug report then?
What kind of NIC do you have and what issue are you running into? I'm running this script on a box with ix and igc interfaces without issues and unaltered just fine.
I have a realtek add-on board:
my experience was much the same as described in this post, the DHCP packet simply wouldn't pass onto the ONT after the initial negotiation.
22.05 has been released. Has anyone been able to test the final release version yet?
I have. Still broken for igb/em interfaces. Works fine on other types of interfaces I've tested.
The base hasn’t changed from FreeBSD 12.3, so does the existing driver solution for 22.01 work? In short, can you safely direct update if you are on 22.01 to 22.05?
If the major kernel version hasn't changed, should be fine.
Rolled the dice and updated. So far so good.
Working fine with 22.05 no problems with both types
For those already using the customer driver, when you upgrade the custom driver stays in place. The root issue is not solved yet but has been marked in Redmine for the "next" release. https://redmine.pfsense.org/issues/12821?next_issue_id=12820
If you do a clean install to 22.05 the issue will probably come back.
I installed opnsense 22.07 and this issue still persists with the Intel interfaces. I changed the config to a Realtek interface and then the script worked.
@neydah700 you saved the day my friend! 2.6 was my first attempt with pfSense. I was about to give up. The above did the trick. I'm up and running and my ATT 1Gb has never been faster. Thank you!
We might finally be able to close this issue out. https://redmine.pfsense.org/issues/12821?next_issue_id=12820
Having issues with ix
Recently updated and all hell broke loose.
Not using
dumped certs,wpa_supplicant,pfSense Plus, etc. just trying to use the original tethered method.I did a backup and clean reinstall + restore, however, now earlyshellcmd is not included, so script can't auto exec on startup, tried symlinking
/usr/local/etc/rc.d/pfatt->/root/pfatt.sh- it runs but still having issues below:After applying PR #73 the issue appears to be at
/usr/sbin/ngctl mkpeer o2m: etf many1 downstreamin the script.uname -a
Error:
Log:
sysctl dev.ix.0
@anthonywww there's no reason to use the tethered method anymore, nor supplicant with netgraph/switch. pfSense includes the necessary patches in wpa_supplicant and dhclient already. All you need are the certificates.
bump https://github.com/MonkWho/pfatt/issues/67#issuecomment-1968192203
If you still want to use the tethered method and not any of the other workarounds you don't need, and probably shouldn't use, netgraph anymore. There is functionality built into pfsense now.
https://docs.netgate.com/pfsense/en/latest/recipes/authbridge.html
Ah yes, I've seen that article, though it doesn't appear to work for the CE (non-Plus) version of PfSense unfortunately. Is there a guide for the CE version?
If there is a guide I never found it. I decided to use the supplicant method since pfsense now supports it in all versions.