OpnSense 22.1 Issue #65
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: hhf/pfatt#65
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
https://opnsense.org/opnsense-22-1-released/
Has anyone had experience with it yet? I haven't gotten around to it and probably won't have time for a while. Starting an issue thread to keep track of it.
EDIT: reported working well with wpa_supplicant
EDIT MORE: https://github.com/MonkWho/pfatt/issues/65#issuecomment-1043984610 seems to be the victorious solution that covers both WPA and Tethered operating modes.
I upgraded to 22.1 without even recalling that I had this script running. Haven't been able to get connection through the gateway since the upgrade. I've verified that the upgrade didn't overwrite/remove the script from
/usr/local/etc/rc.syshook.d/early/. The opnatt log is also showing as everything as having been setup properly. Investigating more now as I'm not sure what would have caused the break with the update.Looks like DHCP traffic is not being bridged between
ngeth0and$ONT_IFnow. Will update as I troubleshoot more...I tried my hand at this over the weekend with trying to use
supplicantmode. When I reboot the system hangs atwaiting EAP for authorization...which is in this while true loop here https://github.com/MonkWho/pfatt/blob/supplicant_OPNsense_testing/bin/opnatt.sh#L212. I'm testing on thesupplicant_OPNsense_testingbranch. I'm trying to get some insight and logging to see if I can figure out what's going on.For some reason none of my logs are being output to
/var/log/opnatt.logor/var/log/pfatt.log. I'm no BSD expert, but/usr/bin/logger -st "pfatt" "starting pfatt..."seems like it would log straight to STDOUT.Edit: I was able to get some better logging through the web ui. It appears something with
dhclientis causing issues. I'm going to try running through the commands in the script one by one to see if I can track down what is going on.Edit Edit: I was able to get this working with the steps below. To me it seems like there is something going on with the order of operations of the script. After a reboot I still get stuck on
waiting EAP for authorization..and have to repeat the steps below. My/etc/wpa_supplicant.confis the one that got generated when I extracted my certs from the device locksmith. So it seems like runningwpa_supplicantdirectly instead of withwpa_cliis making some sort of difference?Move WAN from ngeth0 to em1 in the web gui
sudo /usr/sbin/wpa_supplicant -Dwired -iem1 -c/etc/wpa_supplicant.confMove WAN from em1 to ngeth0 in the web gui
stop wpa_supplicant with CTRL + C
sudo /usr/sbin/wpa_supplicant -Dwired -ingeth0 -c/etc/wpa_supplicant.confstop wpa_supplicant with CTRL + C
sudo /usr/local/etc/rc.syshook.d/early/99-opnattIP4 and IP6 addresses acquired and speed is okay. I'm getting 400/400, but I think it's because my box is running on an old Intel Atom SuperMicro
Intel(R) Atom(TM) CPU D525 @ 1.80GHz (2 cores, 4 threads). Might need to test out some better hardware. Nothing special was done for IP6, it was acquired about a minuet after the IP4I'll have to give this a go. Glad to see its working.
If I recall, when I made my PR to go from wpa_cli to wpa_supplicant, there was some poor escaping or bash interpretation happening in the arguments list. Switching to the .conf file is far more supported anyways in the *nix community at large.
I've got a non-wpa_supplicant setup, so I'm not sure those steps will apply to me. May be a reconfiguration of the interface did something, however.
I have not been able to get any more figured out for non-wpa_supplicant setup, has anyone else made any progress?
Not yet but I'm working on it too. I also tried converting to the wpa supplicant method and can't get it going either but that's true for me on 21.7. If I figure either out I'll report back.
21.7 works for me with supplicant but 22.1 doesn't.
I am also having problems with the latest version of OPNsense, I had to revert to 21.7.8
Since we've got multiple methods of PFATT in this issue, could you please be specific as to which method you're using?
To summarize:
I am using the supplicant method. After the update the scripts hangs on "waiting on EAP for authorization". On occasion running the script manually worked. I decided to move back to the older release of OPNsense since if I had to reboot for any reason the script would not terminate unless it gets a IP address.
Are you using my pull request? Or the script in the repository? I had mine
hang at the same place, but it was the wpa-cli code block that was failing.
There was some bash interpretation and escaping that failed.
Try my pull request script and see if it works.
On Mon, Feb 7, 2022 at 19:19 Hou-dev @.***> wrote:
--
No trees were harmed in the sending of this message, but a rather large
number of electrons were terribly inconvenienced.
I tried the zombielinux fork and it had the same issue.
I'm running in supplicant mode with zombilunix's branch.
@zombielinux I also tried your script and it still hangs at
waiting EAP for authorization..I still have to do the dance of reassigning the WAN port and runningwpa_supplicantmanually. I did have a thought though that maybe the script is running before the interfaces are fully initialized? But that doesn't line up because if I run the script manually after breaking out and letting the system fully boot it hangs at the same spot.I want to try updating the script to manually set the WAN back to
em1, runwpa_supplicant, then once authenticated set the WAN back tongeth0. I have had good luck getting wpa_supplicant to auth againstem1instead ofngeth0. Or maybe it's the other way around? I'm getting inconsistent results in my testing.I was able to get this working with @zombielinux's updates at the encouragement of @MrCaturdayNight
I'm on OPNsense 21.7.8 on protectli hardware ( uses Intel(R) I210 gig copper ports for reference ). I tried running all the commands on the CLI one by one and kept getting erroneous behavior with ngctl, then I tried inserting the kernel module prior to that point in the script explicitly and it worked. So, this is NEARLY identical to the pfatt pull request above. Perhaps the new(er) kernels were removing unused modules (or similar). Interestingly, I noticed the dhcpclient had to iterate over a few retries before success. Not sure if that's relevant or if I simply hadn't seen it run before. Files included / attached for reference:
@dangeist Thanks for the excellent work!
I've updated the pull request with the explicit module loading.
Also, the DHCP taking a few attempts was observed over here too, I'd say its normal behavior.
Purely selfish reasons. I'm moving all my home network gear over to battery-backed DC power and the RGW was a non-standard voltage and power pigtail :)
I tried the updates from @dangeist and @zombielinux and i'm still getting hungup on boot. I should note that i'm running on
22.1and not21.x.After boot i'm able to get authenticated by running
sudo wpa_supplicant -Dwired -ingeth0 /conf/pfatt/wpa/wpa_supplicant.confI'm not sure why it won't auth withem1, my WAN port, but it will withngeth0A big thank you to all of those who are putting work into this. If I find a work around I will report back.
pfatt.sh also breaks on pfSense 2.6.0. I wasn't able to get it to work and I had to downgrade to 2.5.2.
I can also confirm this is broken on pfSense Plus 22.01 and pfSense 2.6.0 CE. Seems something broke in the newer FreeBSD kernel, I'd imagine.
I did some testing last night and it seems with Opnsense 22.1 there are some problems with my setup (Intel nic) with the updated kernel. There is a option to update the kernel only using
opnsense-update -kr 22.1I was able to get authenticated when using my other Broadcom nic but it had issues with its driver dropping packets. I had to revert to use my Intel 350 nic with the older 21.7.8 kernel.Edit 0: I also tried ZombieLinux's version but the same issue with the latest version 22.1 with my Intel nic.
Edit 1: There might be some change in FreeBSD 12.3 that broke compatibility with the script since I am seeing reports that pfsense 2.6 is broken. Opnsense skipped 12.3 and used FreeBSD 13's kernel.
Had the issue this morning when I moved over to 2.6 and ended up trying out OPNsense but haven't tried implementing this, watching closely to look for a solution.
Silver lining is this gave me an opportunity to take a good look at OPNsense and it's pretty nice.
Sounds like a similar issue with us pfSense people. I am a bit lost thought. Has anyone gotten @zombielinux fork working with Opnsense 22.1? Looking at his code it looks like the only major difference between what I was using was is adding "/sbin/kldload -nq ng_ether" at the start of the script. If someone was successful with 22.1 I'll upgrade again and try it on pfSense 22.01.
I can try this later on once the fam goes to bed.
Well this is quite different than with pfsense which I am used to, only been running opnsense for a couple of days now.
When I boot up it says it can't find the file so I don't know what the issue is, when I try and run the file manually using ./filename it says the same thing so IDK.
root@OPNsense:/usr/local/etc/rc.syshook.d/early # ls -l
total 15
-rwxr-xr-x 1 root wheel 135 Jan 25 03:35 05-upgrade
-rwxr-xr-x 1 root wheel 63 Jan 25 03:35 10-configd
-rwxr-xr-x 1 root wheel 77 Jan 25 03:35 15-templates
-rwxr-xr-x 1 root wheel 93 Jan 25 03:35 20-backup
-rwxr-xr-x 1 root wheel 631 Jan 25 03:35 90-carp
-rwxr-xr-x 1 root wheel 3563 Feb 15 17:33 99-opnatt-supplicant
I know this is focused on OpnSense but I was able to do a lot of troubleshooting on the possibly related pfSense issue and may have found a solution. If not a solution for you all hopefully it adds some insight.
https://github.com/MonkWho/pfatt/issues/67#issuecomment-1043358822
And more info on possibly the root issue in FreeBSD
https://github.com/MonkWho/pfatt/issues/67#issuecomment-1043433763
I really do think this is related...
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260068
Unfortunately, it appears than OPNsense has if_em compiled into the kernel. I can confirm their 22.1 branch of the FreeBSD-13 kernel does not have the patch applied...
https://github.com/opnsense/src/tree/stable/22.1/sys/dev/e1000
However, master does...
https://github.com/opnsense/src/tree/stable/13/sys/dev/e1000
I agree. I feel very strongly this is part, if not all of the issues, that have cropped up on opn/pfsense.
Victory!
No custom module or kernel required. Just change this...
/sbin/ifconfig $ONF_IF promiscTo this...
/sbin/ifconfig $ONF_IF promisc -vlanhwtag -vlanhwfilter -vlanhwtsoDid you test this on OPNSense or pfSense? Wpa_supplicant or tethered?
OPNsense with WPA
You got my hopes up! Just tried on pfSense and no luck. Glad it fixes it for you all on OPNSense! Maybe I made a mistake and someone has better luck on pfSense as well!
I also have "VLAN Hardware Filter" to "Disable VLAN Hardware Filtering" in "Interfaces: Settings". I thought this would disable it, but it only seems to act on assigned interfaces... i.e. when the WAN is set to igb0 not ngeth0. So that is why I had to manually set it in the pfatt.sh script.
neydah700, after boot, what does you "ifconfig" look like? You should not see the 3 disabled vlan options. Here's mine...
igb0: flags=28963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500 options=4e027ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>Gotta call it for the night. I'll give it a shot in the morning and let ya know!
Here is mine (pfsense 2.6)
Custom module with switches (no ip)
igb0: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500
options=6003ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,RXCSUM_IPV6,TXCSUM_IPV6>
Default module with switches (no ip)
igb0: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500
options=e007ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
Custom module, tether script (works)
igb0: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500
options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
@lnxsrt
Mine below. Only difference I noticed was the NOMAP at the end of yours.
igb5: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500 options=e027ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,RXCSUM_IPV6,TXCSUM_IPV6>This works for my supplicant setup using an Intel i350. I was able to boot and authenticate.
Has anyone tried this with the "tethered" mode? Apparently Supplicant is popular and maybe I should figure that part out, heh.
Was able to get mine working in tethered mode, now just awaiting a fix for supplicant.
There may be multiple problems here. I had no issue getting WPA to authenticate on the ONT_IF with the same script I was using with 21.7, but no traffic would flow and no DHCP lease could be acquired. Disabling the HW Vlan options seemed to fix the traffic flow issue. So I'd make sure you are getting a wpa auth by using the below command as a first step.
wpa_cli statusWith pfSense I was the same. Never had any issues with WPA. Was just traffic flow on VLAN 0 for DHCP. Unlike you, the script change did not resolve it for me.
Perfect. This worked for me on OPNsense 22.1.x using the supplicant method.
I upgraded from 27.1.8 with this tweak already in my script and my box booted right up, grabbed V4 and V6 and everything seems good so far. I'm on the em driver
Thanks for solving this one.
-vlanhwtag -vlanhwfilter -vlanhwtsoalso worked for me on 22.1.x with supplicant.@zombielinux - Can you make the same change on the opnatt.sh? It worked, I just tested it on a non-supplicant setup and it worked.
Can confirm
/sbin/ifconfig $ONF_IF promisc -vlanhwtag -vlanhwfilter -vlanhwtsoworks with 22.1.x on my protectli hardware using intel NIC and non supplicant setup. I edited the99-pfatt.shscript manually for now. Thanks!@neydah700
same here, no dice, pfsense 2.6
I can confirm changing the script to
/sbin/ifconfig $ONF_IF promisc -vlanhwtag -vlanhwfilter -vlanhwtsoat the bottom of/usr/local/etc/rc.syshook.d/early/99-pfattworks for me as well using non-supplicant method, Opnsense 22.1.2_1 bare metal install on Dell R240 with Intel NICs and BGW-210 AT&T gateway. Survives reboots as always and no throughput or resource usage issues.Are you sure you didn't do any other changes? I spent all weekend trying to get it to work in non-supplicant mode and could not get it to pull a DCHP address for the WAN using a Qotom Q355G4. Clean install of opnsense 22.1.
I also was unable to get it working on non-supplicant 22.1 with just these changes. I have a BGW210 as well.
Honestly, that's the only change I did. I am running in a VM environment w/ NIC passthrough. What NIC does Qotom Q355G4 have?
That's the only thing I did. Keep in mind mine was not a fresh 22.1 install, I already had the bypass up and running prior in 21.7, upgraded to 22.1 in place, bypass didn't work and I simply changed the startup script already present. I never tried changing the initial install script and essentially reinstalling the bypass setup.
It's intel nics. I will have to look up the exact ones later when I get home.
4 x Intel I211-AT
That should just work. Here is a few lines before and after my script. Hopefully it helps you.
I did finally get it to work but it won't work on a reboot of the OPNsense. After a reboot I have to clean up everything the script creates in netgraph then rerun it manually. Then it will pull an IP again just fine. Not sure where to go from there. Seems like I may have an additional issue on top of this one.
what is your netgraph state after the reboot (before the cleanup)?
I am very new to netgraph so could you be more specific in what you are asking for me? Do you just want the output of "ngctl list" or something more?
That would be a good start
I was all prepared to reboot, capture the netgraph output, fix things so I could get back but now it's working perfectly. Three reboots and pulls an IP every time. I do not recall doing anything at that would have made a difference. But as long as it's working now I guess.
Happy to hear, sounds like a timing thing. Sometimes the modem is in an odd spot where it can't pass EAP auth.
For the non-supplicant method, I'm experiencing issues with Opnsense 2.7. I made the required change to the opnatt.sh file, but no change. Anything specific I can look at/test? This is a new build of Opnsense. I'm currently using pfsense 2.5.2 with no issue. One thing I've noticed is the opnatt.sh file - seems several versions out there. I'm using the one in the master branch here. I also require the 5268AC files.
Edit: 3rd restart magic - it works now. Also applied 22.7.2 and we survived. I think another poster above had the same thing - 3 restarts and it works.
I set up Opnsense 22.7 and have tried numerous ways to get this working in supplicant mode. After trying the syntax that dangeist posted, I'm getting the same hanging at waiting EAP for authorization.. How do I get out of this loop? None of the boot options seem to bypass this problem and the loader option doesn't seem to have the options that I need to fix it (modify or delete the script in rc.syshook.d/early)
Thanks!
@SGC1990 Does the /sbin/ifconfig $ONT_IF promisc -vlanhwtag -vlanhwfilter -vlanhwtso fix need to be applied to the supplicant portion of the script? Your commit is only to the bridge section.
Still trying to get supplicant mode working with Opnsense 22.7, but ngeth0 does not have the WAN MAC address and em0 (physical WAN interface) is not set up as promiscuous. When early script is applied, the boot process never completes, so I can't manually change settings as far as I can tell.
Any advice?
Don’t put the script in EARLY. Use START instead. I don’t know if this is a recent change but early scripts run before network startup.
https://docs.opnsense.org/development/backend/autorun.html