We are bitten, again! Fortigate as WCCP client as ASA as WCCP server

After much troubleshooting, tracing and head scratching, we found out why!

Unlike Squid, Fortigate as WCCP client can only do GRE return mode.     ASA’s WCCP server implementation, on the other hand, do not support GRE return mode.

Well, Fortigate as WCCP client can also do both L2 forward & return mode, but ASA can't do L2 mode at all.   

So, in essence, **they** don’t talk.     At least, for the time being, you can’t deploy ASA as WCCP server and have Fortigate as WCCP client.

Note: BlueCoat as a WCCP client can do all of those modes, L2, Return directly to client and GRE return.

Other links of interest

- Technical Note: Configuring WCCP interception of HTTP traffic to a squid proxy using GRE tunneling

- BlueCoat SGOS 6.3 or later's WCCP Reference Guide

- http://www.crypt.gen.nz/papers/cisco_squid_wccp.html (I like the diagram there, essentially showing the "direct-to-client" return method.)
- Cisco Catalyst 6500 Series Switches - WCCP Network Integration with Cisco Catalyst 6500: Best Practice Recommendations for Successful Deployments

“….WCCP redirection is supported only on the ingress of an interface. The only topology that the adaptive security appliance supports is when client and cache engine are behind the same interface of the adaptive security appliance and the cache engine can directly communicate with the client, without going through the adaptive security appliance….”

Ref: http://discussions.citrix.com/topic/330307-redirection-fails-when-using-wccp-on-a-cisco-asa/

FortiManager "Out-of-Sync" Diff View's Deficiency

There are a few improvable part of FortiManager, this is one of them.

When changes are made on Fortigate locally, the FortiManager will detect that change and report “Out-of-Sync” state.    This is well and good.

The idea is good, the execution however can still be improved.       

As an administrator, I would want to verify the change I know I have made locally on the device.     

The “View Diff” feature here is just not good enough, I couldn’t see easily what change I have made, because it reported wrongly changes that I didn’t make and changes that simply is because configs are just mis-aligned.

I seriously did not modify 1210 Line(s).

So, as a result, I need to fly “blind”.   Either Accept wrongly or Reject wrongly.  

Simply is very annoying given that the other part of FortiManager handle the “Diff” so brilliantly such as “ADOM Revisions” Diff.

Running Script Against FortiManager's Policy Package

A CLI script can be applied to the Policy Package’s Object as well as Policy Package’s Policy.    

Use the CLI commands as if you are working with the device locally, but if you have VDOM in place, just remove that part.

Instead of using

config vdom

edit “VDOM1”

…list of command…

Simply remove the VDOM reference, and leave the rest of the command.

On the script page, just use “Advanced - Override Script Target” and change the Target to Run on “Policy Package, ADOM Database or Global Policy”.

FortiManager Bug - "+" sign in Web Filtering Profile Name

There is a bug in FortiManager, when ‘+’ sign is used for Web Filtering Profile Name.    “Where Used” function is not working correctly.

Once the plus sign is removed from the profile name, it works correctly.

FortiManager - Import Policy from Device

Why do I need this?

Sometimes it is easier to make change locally on Fortigate.    In my case, I want to create a “ftp-proxy” rule on Fortigate because I seems that the rule cause a problem when install to the device if it is implemented on FortiManager.  

Step by Step

First, lock ADOM

Then, right click on device, select “Import Policy”

Then, select on the VDOM that change has been made, and skip the rest.   Place the policy in a new TEMP POLICY PACKAGE so that it doesn’t mix up with our existing POLICY PACKAGE.  To save space, I won’t show the screen shot of every screen.   I just click next through until finish and click “Skip Remaining”.

After that we can change it the way we like.

If you have created new objects locally on the Fortigate VDOM, it will be imported at this step.

Now, rule #5 is imported as I wish.

From a temporary POLICY PACKAGE “VUFG1_VUWF101” , I just need to COPY & PASTE it into my production POLICY PACKAGE “VU-WEBFILTER-PACKAGE”.

Clean up steps

Now, we need to clean up a few things.

First, place the VDOM back to production POLICY PACKAGE.

Click on Policy & Objects -> YOUR TEMPORARY POLICY PACKAGE -> ‘Install’ Tab -> Right click on your VDOM, then Edit

Select the VDOMs that will use this POLICY PACKAGE.

Remove the VDOM from this temporary POLICY PACKAGE.

And, it will be removed from the list

Check the production POLICY PACKAGE, ‘Install’ Tab, it should be there.

Push to device

Now, the next step is to click on SAVE and then push the revise POLICY PACKAGE to the VDOMs using “Install Wizard”.

Then, on Device tab, it should display “SYNCHRONIZED”

The last step is to simply Unlock VDOM.

Fortigate as WCCP client - WCCP primary-hash by src-ip dst-ip src-port dst-port

The ‘primary-hash’ configuration must match on all of the WCCP client pool. 

At first, I changed it to VDOM2, WCCP server then drop VDOM2 from the pool and assign 100% load to VDOM1 which the change hasn’t been made.

Only after I changed both VDOMs that it starts to INITIALIZE WCCP Pool again (around 15 - 30 seconds), then both become available again.

Cisco-IOS#show ip wccp 70 detail
WCCP Client information:
        WCCP Client ID:          x.x.x.x
        Protocol Version:        2.0
        State:                   Usable
        Redirection:             GRE
        Packet Return:           GRE
        Assignment:              HASH
        Initial Hash Info:       00000000000000000000000000000000
                                 00000000000000000000000000000000
        Assigned Hash Info:      48922489224891244912449124491244
                                 91244912449224891244912449124491
        Hash Allotment:          77 (30.07%)
        Packets s/w Redirected:  4
        Connect Time:            00:00:45
        Bypassed Packets
          Process:               0
          CEF:                   0
          Errors:                0

        WCCP Client ID:          y.y.y.y
        Protocol Version:        2.0
        State:                   Usable
        Redirection:             GRE
        Packet Return:           GRE
        Assignment:              HASH
        Initial Hash Info:       00000000000000000000000000000000
                                 00000000000000000000000000000000
        Assigned Hash Info:      B76DDB76DDB76EDBB6EDBB6EDBB6EDBB
                                 6EDBB6EDBB6DDB76EDBB6EDBB6EDBB6E
        Hash Allotment:          179 (69.92%)
        Packets s/w Redirected:  164
        Connect Time:            00:00:41
        Bypassed Packets
          Process:               0
          CEF:                   0
          Errors:                0

Config on VDOM1

VUFG1 # config vdom

VUFG1 (vdom) # edit VUWF101
current vf=VUWF101:2

VUFG1 (VUWF101) # show system wccp
path=system, objname=wccp, tablename=(null), size=228
config system wccp
    edit "70"
        set cache-id x.x.x.x
        set router-list "z.z.z.z"
        set ports 80 443
        set primary-hash src-ip dst-ip src-port dst-port
        set protocol 6
        set assignment-weight 30
    next
end

Config on VDOM2

VUFG1 # config vdom

VUFG1 (vdom) # edit VUWF102
current vf=VUWF102:1

VUFG1 (VUWF102) # show system wccp
path=system, objname=wccp, tablename=(null), size=228
config system wccp
    edit "70"
        set cache-id y.y.y.y
        set router-list "z.z.z.z"
        set ports 80 443
        set primary-hash src-ip dst-ip src-port dst-port
        set protocol 6
        set assignment-weight 70
    next
end

Fortigate as WCCP client - Adjusting assignment weight between WCCP clients in the pool

From Cisco IOS Viewpoint

Cisco-IOS#show ip wccp 70 detail
WCCP Client information:
        WCCP Client ID:          x.x.x.x
        Protocol Version:        2.0
        State:                   Usable
        Redirection:             GRE
        Packet Return:           GRE
        Assignment:              HASH
        Initial Hash Info:       00000000000000000000000000000000
                                 00000000000000000000000000000000
        Assigned Hash Info:      7FF7FFBFFDFFEFFEFFF7FFBFFDFFDFFE
                                 FFF7FFBFFBFFDFFEFFF7FF7FFBFFDFFD
        Hash Allotment:          235 (91.79%)
        Packets s/w Redirected:  1337
        Connect Time:            22:36:58
        Bypassed Packets
          Process:               0
          CEF:                   0
          Errors:                0

        WCCP Client ID:          x.x.x.y
        Protocol Version:        2.0
        State:                   Usable
        Redirection:             GRE
        Packet Return:           GRE
        Assignment:              HASH
        Initial Hash Info:       00000000000000000000000000000000
                                 00000000000000000000000000000000
        Assigned Hash Info:      80080040020010010008004002002001
                                 00080040040020010008008004002002
        Hash Allotment:          21 (8.20%)
        Packets s/w Redirected:  1282
        Connect Time:            21:56:44
        Bypassed Packets
          Process:               0
          CEF:                   0
          Errors:                0

From VDOM1

VUFG1 # config vdom

VUFG1 (vdom) # edit VUWF101
current vf=VUWF101:2

VUFG1 (VUWF101) # show system wccp
path=system, objname=wccp, tablename=(null), size=228
config system wccp
    edit "70"
        set cache-id x.x.x.x
        set router-list "z.z.z.z"
        set ports 80 443
        set protocol 6
        set assignment-weight 235
    next
end

From VDOM2

VUFG1 # config vdom

VUFG1 (vdom) # edit VUWF102
current vf=VUWF102:1

VUFG1 (VUWF102) # show system wccp
path=system, objname=wccp, tablename=(null), size=228
config system wccp
    edit "70"
        set cache-id x.x.x.y
        set router-list "z.z.z.z"
        set ports 80 443
        set protocol 6
        set assignment-weight 20
    next
end

What If VDOM1 stop WCCP?

After a few seconds, the HASH allocation change to 100% on VDOM2

Cisco-IOS#show ip wccp 70 detail
WCCP Client information:
        WCCP Client ID:          x.x.x.y
        Protocol Version:        2.0
        State:                   Usable
        Redirection:             GRE
        Packet Return:           GRE
        Assignment:              HASH
        Initial Hash Info:       00000000000000000000000000000000
                                 00000000000000000000000000000000
        Assigned Hash Info:      80080040020010010008004002002001
                                 00080040040020010008008004002002
        Hash Allotment:          21 (8.20%)
        Packets s/w Redirected:  1282
        Connect Time:            22:01:43
        Bypassed Packets
          Process:               0
          CEF:                   0
          Errors:                0

Cisco-IOS#show ip wccp 70 detail
WCCP Client information:
        WCCP Client ID:          x.x.x.y
        Protocol Version:        2.0
        State:                   Usable
        Redirection:             GRE
        Packet Return:           GRE
        Assignment:              HASH
        Initial Hash Info:       00000000000000000000000000000000
                                 00000000000000000000000000000000
        Assigned Hash Info:      FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
                                 FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
        Hash Allotment:          256 (100.00%)
        Packets s/w Redirected:  1282
        Connect Time:            22:01:45
        Bypassed Packets
          Process:               0
          CEF:                   0
          Errors:                0

What Happens if the Hash Weight doesn't add up to 255?

Assignment will be done in porportion

In this case, VDOM1 wieght is 100, VDOM2 weight is 20.   VDOM1 will get 100/120, and VDOM2 will get 20/120 of the traffic.

VUFG1 # config vdom

VUFG1 (vdom) # edit VUWF101
current vf=VUWF101:2

VUFG1 (VUWF101) # show system wccp
path=system, objname=wccp, tablename=(null), size=228
config system wccp
    edit "70"
        set cache-id x.x.x.x
        set router-list "z.z.z.z"
        set ports 80 443
        set protocol 6
        set assignment-weight 235
    next
end

Cisco-IOS#show ip wccp 70 detail
WCCP Client information:
        WCCP Client ID:          x.x.x.x
        Protocol Version:        2.0
        State:                   Usable
        Redirection:             GRE
        Packet Return:           GRE
        Assignment:              HASH
        Initial Hash Info:       00000000000000000000000000000000
                                 00000000000000000000000000000000
        Assigned Hash Info:      EFBEFBEFBEFBEFBEFBEFBEFBEFBEFBEF
                                 BEFBEFBEFBEFBEFBEFBEFBEFBEFBEFBD
        Hash Allotment:          213 (83.20%)
        Packets s/w Redirected:  0
        Connect Time:            00:00:30
        Bypassed Packets
          Process:               0
          CEF:                   0
          Errors:                0

        WCCP Client ID:          x.x.x.y
        Protocol Version:        2.0
        State:                   Usable
        Redirection:             GRE
        Packet Return:           GRE
        Assignment:              HASH
        Initial Hash Info:       00000000000000000000000000000000
                                 00000000000000000000000000000000
        Assigned Hash Info:      10410410410410410410410410410410
                                 41041041041041041041041041041042
        Hash Allotment:          43 (16.79%)
        Packets s/w Redirected:  1285
        Connect Time:            22:05:36
        Bypassed Packets
          Process:               0
          CEF:                   0
          Errors:                0

Live Sample - Retrieve Device's System Config from FortiManager

After a necessary change is made on the device, for example, in my case WCCP client config, FortiManager detects that the configs are now out of sync.

config system interface

edit port3

set wccp enable

end

config system settings

    set wccp-cache-engine enable

end

config system wccp

    edit "70"

        set cache-id x.x.x.x

        set router-list "y.y.y.y"

        set ports 80 443

        set protocol 6

    next

end

To get it to be in-sync again, the best way is not to use this “Accept” diaglog, I think the retrieve config is a better way.

To do that, Lock the Device’s ADOM

Click on Revision History, then Click on “Retrieve”.

Wait while the config is being retrieved.

Then, push it back to the device, to get the “in sync” status again.

The device is now in “Sync” again.

Fortiguard Category Number (For FortiOS 5.0)

FortiGuard Category Number for FortiOS5.0

REF: Fortiguard Web Filtering Group, https://www.fortiguard.com/static/webfiltering.html

Group: Security Risk

Malicious Websites, 26

Phishing, 61

Spam URLs, 86

Group: General Interest - Business

Finance and Banking, 31

Search Engines and Portals, 41

General Organizations, 43

Business, 49

Information and Computer Security, 50

Government and Legal Organizations, 51

Information Technology, 52

Armed Forces, 53

Web Hosting, 81

Secure Websites, 56

Web-based Applications, 84

Group: Adult/Mature Content

Alternative Beliefs, 2

Abortion, 7

Other Adult Materials, 8

Advocacy Organizations, 9

Gambling, 11

Nudity and Risque, 13

Pornography, 14

Dating, 15

Weapons (Sales), 16

Marijuana, 57

Sex Education, 63

Alcohol, 64

Tobacco, 65

Lingerie and Swimsuit, 66

Sports Hunting and War Games, 67

Group: Bandwidth Consuming

Freeware and Software Downloads, 19

File Sharing and Storage, 24

Streaming Media and Download, 25

Peer-to-peer File Sharing, 72

Internet Radio and TV, 75

Internet Telephony, 76

Group: Potentially Liable

Drug Abuse, 1

Hacking, 3

Illegal or Unethical, 4

Discrimination, 5

Explicit Violence, 6

Extremist Groups, 12

Proxy Avoidance, 59

Plagiarism, 62

Child Abuse, 83

Group: General Interest - Personal

Advertising, 17

Brokerage and Trading, 18

Games, 20

Web-based Email, 23

Entertainment, 28

Arts and Culture, 29

Education, 30

Health and Wellness, 33

Job Search, 34

Medicine, 35

News and Media, 36

Social Networking, 37

Political Organizations, 38

Reference, 39

Global Religion, 40

Shopping and Auction, 42

Society and Lifestyles, 44

Sports, 46

Travel, 47

Personal Vehicles, 48

Dynamic Content, 54

Meaningless Content, 55

Folklore, 58

Web Chat, 68

Instant Messaging, 69

Newsgroups and Message Boards, 70

Digital Postcards, 71

Child Education, 77

Real Estate, 78

Restaurant and Dining, 79

Personal Websites and Blogs, 80

Content Servers, 82

Domain Parking, 85

Personal Privacy, 87

Group: Unrated

Unrated, 56

Finding Fortiguard Category Number (For FortiOS 5.0)

FortiOS 4.x

FortiOS 4.x has category number published here - http://kb.fortinet.com/kb/documentLink.do?externalID=FD30715. As of this writing, this is not applicable to FortiOS 5.x because there is no “explanation text” as describe.

Step 1

Enable cli debugging

config vdom

edit VUWF101

diag debug cli 8

diag debug enable

Step 2

Turned on “monitor” on the category one at a time.  Click Apply

Step 3.

Part of the CLI debug output would identify the corresponding category number.

0: config webfilter profile

0: edit "test2"

0: config ftgd-wf

0: config filters

0: edit 2

0: set category 19

0: end

0: end

0: end

In this case, Freeware and Software Download is category number 19