FWP_E_INCOMPATIBLE_AUTH_METHOD 0x80320030 Fix

You're trying to apply IPsec rules with an auth method that doesn't match the policy type. Drop the certificate requirement or switch to a compatible policy.

Quick answer: The auth method you specified (usually certificate or NTLM) isn't allowed by the policy type you're creating (usually an authenticated bypass or a specific tunnel rule). Change the auth method or change the policy type — they have to match.

FWP_E_INCOMPATIBLE_AUTH_METHOD shows up when Windows Filtering Platform rejects a rule because the authentication method doesn't fit the policy being defined. You'll see it most often when you're scripting IPsec rules with New-NetIPsecRule or pushing Connection Security Rules through GPO. The classic trigger: someone copies a working certificate-based auth rule into a policy that only supports Kerberos, or tries to add a certificate auth to a tunnel rule that expects user-level auth. WFP validates the combination up front and rejects the whole thing. No partial apply — the rule just fails.

What's actually happening

Every IPsec rule in Windows has two layers: the policy type and the auth method. Policy types include things like Transport, Tunnel, Authenticated Bypass, and Isolation. Auth methods include Kerberos v5, NTLMv2, computer certificates, user certificates, and pre-shared keys. Not every auth method works with every policy type — for example, Authenticated Bypass only accepts computer-level auth (Kerberos or computer certificate). Try to bolt on user certificate auth and WFP throws 0x80320030 at you.

Common real-world scenario: a junior admin migrates firewall rules from a legacy server that used PSK (pre-shared key) into a new GPO that's configured for domain isolation. The GPO only allows Kerberos. Every rule with PSK fails, and the event log fills up with 0x80320030. Took me an afternoon to untangle the first time I hit it, back when Windows Server 2008 R2 was current.

Step-by-step fix

Identify the failing rule. Check PowerShell output or the Event Viewer. Look under Applications and Services Logs > Microsoft > Windows > Windows Filtering Platform > Operational. Filter for event ID 5447 or 5440. The XML payload names the rule and the rejected auth method.
Check the policy type. Run this to see the current rule definition:
```
Get-NetIPsecRule -DisplayName "YourRuleName" | Select DisplayName, PolicyStore, AuthMethod, PrimaryStatus
```
If AuthMethod is something like ComputerCertificate but the rule lives under an isolation policy, that's your mismatch.
Decide which side to change. Either the auth method or the policy type has to give. If the environment is domain-joined and Kerberos works, switch auth to Kerberos — that's usually the easy path. If you genuinely need certificates (say, for a non-domain server), the policy type has to support certificate auth, which means it can't be an authenticated bypass.
Rebuild the rule with matching parameters. Example for a transport-mode rule using Kerberos:
```
New-NetIPsecRule -DisplayName "Server Transport Rule" `
  -InboundSecurity Require `
  -OutboundSecurity Require `
  -Protocol TCP `
  -LocalPort 445 `
  -RemoteAddress Any `
  -AuthMethod Kerberos `
  -PolicyStore ActiveStore
```
Note the -AuthMethod value matches what the policy type allows. If you leave it off, Windows picks a default based on the store's defaults, which may or may not match.
Verify. Run Get-NetIPsecRule again, or check netsh advfirewall consec show rule name=all. The PrimaryStatus field should read OK, not NotApplicable or NotRunning.

If that doesn't work

Sometimes the rule looks fine but WFP still rejects it. Here's what else to check:

Merged policies. If you've got both local and GPO-sourced rules, they can conflict. Check Get-NetIPsecRule -PolicyStore ActiveStore vs PersistentStore. ActiveStore is the merged view. If the local rule uses PSK and the GPO demands Kerberos, the merge fails silently for that rule with 0x80320030.
Certificate store issues. If you're using certificate auth legitimately, the certificate has to be in the right store with the right EKU. A missing or expired cert can manifest as the same error in some builds because WFP can't validate the auth method source.
Domain isolation policy version. Older isolation policies inherited from Windows 7-era GPOs sometimes hardcode NTLM as a fallback. If NTLM is disabled in your domain, that fallback triggers 0x80320030. Audit the GPO's Connection Security Rules section.
Pre-shared key restrictions. PSK is deprecated and some newer builds restrict it under certain policy types. If you're still using PSK, that's a security problem anyway — move to Kerberos or certs.

Don't bother running netsh winsock reset or re-registering WFP DLLs for this one. It's a configuration mismatch, not a corrupted stack. I've seen people chase that rabbit hole for hours and it never helps.

Prevention

Define your auth method standard before you write rules. Pick one of these and stick to it: Kerberos for domain-joined assets, certificates for everything else. Document which policy types support which auth methods in your environment — authenticated bypass and isolation rules are the ones that bite people. And when you copy rules between policies or environments, always re-check the auth method. That's where 0x80320030 comes from 90% of the time.

Rule of thumb: if the rule works on server A but fails on server B with 0x80320030, the two servers are in different policy scopes or have different GPO defaults. Compare Get-NetIPsecRule -PolicyStore ActiveStore output side by side.