Lazyness at your service!: Automated load of object through API from CSV!

Disclaimer: This information is posted as-is and the content should be referenced at your own risk


DISCLAIMER: Im not a programmer, not even close, everything here is self taught, might containt errors, test it in your environment prior to using in real life. Always make backups, TEST the backups and export full configurations.

Updated 21 November, Version 1.5:

  • Fixed some things (FQDN detection)
  • Added credential validation

Hey!

I think lazyness its the mother of inventive. I prefer to use 1 hour to create automated things to view more cats memes study more than doing manual stuff. Thats why I did a little script to help load mostly IPHosts and FQDN. Helped me a lot when migrating firewalls from other brands since objects its the more boring stuff to migrate.

What do you need?

  • One computer in which you can run a Powershell script
  • A list of things to upload to your firewall
  • A firewall (duh!). Ideally Sophos, but it works for Palo Alto (!!)
  • API Enabled for the computer

What can you with this?

In IPHost "mode":

You can feed a CSV file with the following format:

Object name,IP addres,netmask

Examples

Gateway IP,192.168.0.254,/27
Printer,172.31.0.15,255.255.255.255
Office AP,172.16.0.10
Web Server,10.10.17.45,24

And he'll feed the firewall and create the appropiate objects. It can read the mask with the following formats:

  • /XX (Slash and a number, /24, /16, /30, etc)
  • XX (just two numbers, 24, 16, 30, etc)
  • xxx.xxx.xxx.xx (normal representation like 255.255.255.0)
  • No mask at all (it will assume that its a host ip with /32 mask)

In FQDN "mode":

You can feed a CSV file with the following format:

Object name, FQDN

You can choose to omit the FQDN and in that case it will use the name as the FQDN.

Examples

Google,*.google.com
*.amazon.com
www.ebay.com,*.ebay.com

In Automode (TM) BETA AF

You can feed a CSV file with a mix of IPHost/FQDN and the script will interpret (99,9% of the times) the appropiate object.

Examples

*.amazon.com
Gateway IP,192.168.0.254,/27

Printer,172.31.0.15,255.255.255.255
Office AP,172.16.0.10
*.microsoft.com
NASA Website,90.43.1.65

domain.localhost.com
NASA Site FQDN,*.nasa.gov

Assumptions the script does in this mode:

  • If it has 1 field (ex: *.amazon.com), its considered a FQDN (working to improve detection for more cases)
  • If it has 2 or more fields, it will check if the second field matches an IP address format (xxx.xxx.xxx.xxx), if it does, its considered an IPHost and same rules for IPHost applies here (if it doesn't have netmask, its assumed an IPHost /32). If the second field doesnt match that, its considered a FQDN host.

How can I use it?

You need to download the attached .PS1 file and put in somewhere your Windows computer. Then you have to open "Windows Powershell ISE" in administrator mode and open the script in there.

Now, go to your firewall and allow API usage from the IP that will have the script running:

  1. Enable API
  2. Type the IP address of the computer hosting the script
  3. Add it
  4. Apply

Once you have opened the script, you need to supply the parameters for your firewall:

  1. The IP of your firewall
  2. The admin port of your firewall
  3. The admin user (normally "admin")
  4. The password for the account
  5. The name of the file including the objects (needs to be inside "work folder")
  6. The work folder (any folder as long as the script can write there)

And then, run it!

You'll be prompted with the execution mode as explained before. Be careful, if you select "IPHost" but you have FQDN in your file, it might cause problems or errors. 

Choose your mode and press ENTER. The program will start checking every object and proceed to create it in the firewall and provide output about that operation. If the code is 200, everything is ok. Otherwise, check the code in the Sophos API documentation.

The script also creates a log file in the working folder for future references about what failed and what ended up OK.

Whats next?

There're some know issues:

  • If you mess up the password, it will fail every item in the file instead of saying it at the beginning, working on it.
  • Trying to add more ways for automode to accept FQDN (check for *?, check for 255?, etc).
  • Lacking some arguments to run it in non interactive mode (faster?)
  • Will try to add more things that it can automate (services?, iphostgroups?)

And yeah, basically this is a hobby for fun but if you have doubts, requests, bugs, let me know. I'll try to address them.

Take care, stay safe!

4540.ImportMan.ps1.txt
# ---------------------
# ImportMan - XG
# ---------------------
# Versions
#
# 1.0 - Version original
# 1.1 - Added code to allow different masks in CSV
# 1.2 - Translated to English from Spanish
# 1.3 - Multi version - works with fqdn too
# 1.4 - Improved iphost detection, fqdn detection. Added automode TM
# 1.5 - Improved FQDN detection and cases available. Validation of credentials prior to creationg of objects. Minor translations.
#
# ---------------------

# Functions
# ---------



# Global variable declaration
# ------------------------------
param ($_OPERATION)
$_AUTOMODE = 0
$_FIREWALL_IP = "172.31.0.254"
$_FIREWALL_PORT = "4444"
$_API_USER = "admin"
$_API_PASSWORD = 'leroLERO1407$$$'
$_WORK_FOLDER = "D:\"
$_CSV_FILE_NAME = "text.xml"
$_LOGS_FOLDER = ([String](Get-Date -Format 'ddMMyyyy_HHmm'))

New-Item -ItemType "directory" -Path $_WORK_FOLDER -Name $_LOGS_FOLDER | Out-Null

# HTTPutility addon
# -------------------------
Add-Type -AssemblyName System.Web

# Convert variable to URI compatible format
# -----------------------
$_CODIFIED_API_PASSWORD = [System.Web.HttpUtility]::UrlEncode($_API_PASSWORD)

# Allow SSL connections
# -----------------------
add-type @"
    using System.Net;
    using System.Security.Cryptography.X509Certificates;
    public class TrustAllCertsPolicy : ICertificatePolicy {
        public bool CheckValidationResult(
            ServicePoint srvPoint, X509Certificate certificate,
            WebRequest request, int certificateProblem) {
            return true;
        }
    }
"@

[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 

# Check operation to be performed
if ($_OPERATION -eq $null) 
    {
    Write-Host ""
    Write-Host ""
    Write-Host "ImportMAN! 1.5"
    Write-Host "--------------"
    Write-Host ""
    Write-Host "Helping YOU stay lazy."
    Write-Host ""
    Write-Host ""
    Write-Host "Type:"
    Write-Host ""
    Write-Host "1 for IPHost mode"
    Write-Host "2 for FQDN mode"
    Write-Host "3 for Auto-Mode(TM) BETA AF mode"
    Write-Host ""
    Write-Host ""
    $_OPERATION = read-host -Prompt "Please select operation mode: "
	if ($_OPERATION -eq 3) {$_AUTOMODE = 1}
    Write-Host ""
    Write-Host ""
    }

# Testing credentials prior to upload objects
# -------------------------------------------
$_URL_API = "https://$($_FIREWALL_IP):$($_FIREWALL_PORT)/webconsole/APIController?reqxml=<Request><Login><UserName>$($_API_USER)</UserName><Password>$($_CODIFIED_API_PASSWORD)</Password></Login></Request>"
$_API_RESULT = Invoke-WebRequest -Uri "$_URL_API"
[xml] $_XML_CODE = $_API_RESULT.Content

if ($_XML_CODE.Response.Login.Status -ne "Authentication Successful")
    {
    Write-Host "[WARN] A problem have occur connecting to the firewall or credentials are not valid. Check and try again later."
    break
    } else {
    Write-Host "[ OK ] Connection to firewall succesful. Starting object creation."
    Write-Host ""
    }

# Main function
# ---------------
foreach($line in [System.IO.File]::ReadLines("$_WORK_FOLDER$_CSV_FILE_NAME"))
    {
    # Obtain and split CSV data
    # ------------------------
    $_ITEM_NAME = ($line -split ',')[0]
    $_SECOND_FIELD =  ($line -split ',')[1]
    $_THIRD_FIELD =  ($line -split ',')[2]  

    # Checking for empty line
    # ------------------------
    if ($_ITEM_NAME -eq $null)
        {
        Write-Host "[INFO] Skipping empty line."
        Continue
        }

    # Detection of type of object for AutoMode
    # ------------------------
	if ($_AUTOMODE -eq 1)
		{
		if ($_SECOND_FIELD -eq $null) 
			{
			$_OPERATION = 2
			} else {
            if ( $_SECOND_FIELD -match '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}')
                {
                $_OPERATION = 1
                } else {
                $_OPERATION = 2
                }
			}
		}
		
    # Netsmak conversion from CSV data
    # ------------------------
    if ($_OPERATION -eq 1)
        {     
	    if (($_THIRD_FIELD -eq "/32") -or ($_THIRD_FIELD -eq "32")){ $_THIRD_FIELD = "255.255.255.255" }
	    if (($_THIRD_FIELD -eq "/31") -or ($_THIRD_FIELD -eq "31")){ $_THIRD_FIELD = "255.255.255.254" }
	    if (($_THIRD_FIELD -eq "/30") -or ($_THIRD_FIELD -eq "30")){ $_THIRD_FIELD = "255.255.255.252" }
	    if (($_THIRD_FIELD -eq "/29") -or ($_THIRD_FIELD -eq "29")){ $_THIRD_FIELD = "255.255.255.248" }
	    if (($_THIRD_FIELD -eq "/28") -or ($_THIRD_FIELD -eq "28")){ $_THIRD_FIELD = "255.255.255.240" }
	    if (($_THIRD_FIELD -eq "/27") -or ($_THIRD_FIELD -eq "27")){ $_THIRD_FIELD = "255.255.255.224" }
	    if (($_THIRD_FIELD -eq "/26") -or ($_THIRD_FIELD -eq "26")){ $_THIRD_FIELD = "255.255.255.192" }
	    if (($_THIRD_FIELD -eq "/25") -or ($_THIRD_FIELD -eq "25")){ $_THIRD_FIELD = "255.255.255.128" }
	    if (($_THIRD_FIELD -eq "/24") -or ($_THIRD_FIELD -eq "24")){ $_THIRD_FIELD = "255.255.255.0" }
	    if (($_THIRD_FIELD -eq "/23") -or ($_THIRD_FIELD -eq "23")){ $_THIRD_FIELD = "255.255.254.0" }
	    if (($_THIRD_FIELD -eq "/22") -or ($_THIRD_FIELD -eq "22")){ $_THIRD_FIELD = "255.255.252.0" }
	    if (($_THIRD_FIELD -eq "/21") -or ($_THIRD_FIELD -eq "21")){ $_THIRD_FIELD = "255.255.248.0" }
	    if (($_THIRD_FIELD -eq "/20") -or ($_THIRD_FIELD -eq "20")){ $_THIRD_FIELD = "255.255.240.0" }
	    if (($_THIRD_FIELD -eq "/19") -or ($_THIRD_FIELD -eq "19")){ $_THIRD_FIELD = "255.255.224.0" }
	    if (($_THIRD_FIELD -eq "/18") -or ($_THIRD_FIELD -eq "18")){ $_THIRD_FIELD = "255.255.192.0" }
	    if (($_THIRD_FIELD -eq "/17") -or ($_THIRD_FIELD -eq "17")){ $_THIRD_FIELD = "255.255.128.0" }
	    if (($_THIRD_FIELD -eq "/16") -or ($_THIRD_FIELD -eq "16")){ $_THIRD_FIELD = "255.255.0.0" }
	    if (($_THIRD_FIELD -eq "/15") -or ($_THIRD_FIELD -eq "15")){ $_THIRD_FIELD = "255.254.0.0" }
	    if (($_THIRD_FIELD -eq "/14") -or ($_THIRD_FIELD -eq "14")){ $_THIRD_FIELD = "255.252.0.0" }
	    if (($_THIRD_FIELD -eq "/13") -or ($_THIRD_FIELD -eq "13")){ $_THIRD_FIELD = "255.248.0.0" }
	    if (($_THIRD_FIELD -eq "/12") -or ($_THIRD_FIELD -eq "12")){ $_THIRD_FIELD = "255.240.0.0" }
	    if (($_THIRD_FIELD -eq "/11") -or ($_THIRD_FIELD -eq "11")){ $_THIRD_FIELD = "255.224.0.0" }
	    if (($_THIRD_FIELD -eq "/10") -or ($_THIRD_FIELD -eq "10")){ $_THIRD_FIELD = "255.192.0.0" }
	    if (($_THIRD_FIELD -eq "/9") -or ($_THIRD_FIELD -eq "9")){ $_THIRD_FIELD = "255.128.0.0" }
	    if (($_THIRD_FIELD -eq "/8") -or ($_THIRD_FIELD -eq "8")){ $_THIRD_FIELD = "255.0.0.0" }
	    if (($_THIRD_FIELD -eq "/7") -or ($_THIRD_FIELD -eq "7")){ $_THIRD_FIELD = "254.0.0.0" }
	    if (($_THIRD_FIELD -eq "/6") -or ($_THIRD_FIELD -eq "6")){ $_THIRD_FIELD = "252.0.0.0" }
	    if (($_THIRD_FIELD -eq "/5") -or ($_THIRD_FIELD -eq "5")){ $_THIRD_FIELD = "248.0.0.0" }
	    if (($_THIRD_FIELD -eq "/4") -or ($_THIRD_FIELD -eq "4")){ $_THIRD_FIELD = "240.0.0.0" }
	    if (($_THIRD_FIELD -eq "/3") -or ($_THIRD_FIELD -eq "3")){ $_THIRD_FIELD = "224.0.0.0" }
	    if (($_THIRD_FIELD -eq "/2") -or ($_THIRD_FIELD -eq "2")){ $_THIRD_FIELD = "192.0.0.0" }
	    if (($_THIRD_FIELD -eq "/1") -or ($_THIRD_FIELD -eq "1")){ $_THIRD_FIELD = "128.0.0.0" }
	    if (($_THIRD_FIELD -eq "/0") -or ($_THIRD_FIELD -eq "0")){ $_THIRD_FIELD = "0.0.0.0" }
        if ($_THIRD_FIELD -eq $null){ $_THIRD_FIELD = "255.255.255.255" }
        }
	
    # Operations for FQDN selection
	# -----------------------------
    
    if ($_OPERATION -eq 2)
        {
        if ($_SECOND_FIELD -eq $null)
            {
            $_SECOND_FIELD = $_ITEM_NAME
            }
	    }
	
    # Convert CSV variable to URI compatible format
    # ---------------------------------------------
    $_CODIFIED_ITEM_NAME = [System.Web.HttpUtility]::UrlEncode($_ITEM_NAME)
    $_CODIFIED_ITEM_NAME = ($_CODIFIED_ITEM_NAME -replace "%26","%26amp;")
        
    # API Query
    # ---------
	
	# For IPHost item
	# ----------------
	if ($_OPERATION -eq 1)
	    {
		if($_THIRD_FIELD -eq "255.255.255.255")
			{
			$_URL_API = "https://$($_FIREWALL_IP):$($_FIREWALL_PORT)/webconsole/APIController?reqxml=<Request><Login><UserName>$($_API_USER)</UserName><Password>$($_CODIFIED_API_PASSWORD)</Password></Login><Set><IPHost><Name>$($_CODIFIED_ITEM_NAME)</Name><IPFamily>IPv4</IPFamily><HostType>IP</HostType><IPAddress>$($_SECOND_FIELD)</IPAddress></IPHost></Set></Request>"
			$_TYPE_OBJECT = "Host"
			} else {
			$_URL_API = "https://$($_FIREWALL_IP):$($_FIREWALL_PORT)/webconsole/APIController?reqxml=<Request><Login><UserName>$($_API_USER)</UserName><Password>$($_CODIFIED_API_PASSWORD)</Password></Login><Set><IPHost><Name>$($_CODIFIED_ITEM_NAME)</Name><IPFamily>IPv4</IPFamily><HostType>Network</HostType><IPAddress>$($_SECOND_FIELD)</IPAddress><Subnet>$($_THIRD_FIELD)</Subnet></IPHost></Set></Request>"
			$_TYPE_OBJECT = "Network"
			}  
	    }
	
	# For FQDN item
	# --------------
	if ($_OPERATION -eq 2)
	    {
		$_URL_API = "https://$($_FIREWALL_IP):$($_FIREWALL_PORT)/webconsole/APIController?reqxml=<Request><Login><UserName>$($_API_USER)</UserName><Password>$($_CODIFIED_API_PASSWORD)</Password></Login><Set><FQDNHost><Name>$($_CODIFIED_ITEM_NAME)</Name><FQDN>$($_SECOND_FIELD)</FQDN></FQDNHost></Set></Request>"
        $_TYPE_OBJECT = "FQDN"		
	    }

    Write-Host ""
	Write-Host "[INFO] Processing $($_TYPE_OBJECT)"
    Write-Host "[INFO] Name: $($_ITEM_NAME)"
    Write-Host "[INFO] Value: $($_SECOND_FIELD)" 
    
    $_API_RESULT = Invoke-WebRequest -Uri "$_URL_API"
    [xml] $_XML_CODE = $_API_RESULT.Content


    # Check result
    # ------------
    $_LINEA_LOG = [String](Get-Date) + "," + "[$($_TYPE_OBJECT)]" + "," + $_ITEM_NAME

    # For IPHost item
	# ----------------
    if ($_OPERATION -eq 1)
        {
        if($_XML_CODE.Response.IPHost.Status.code -eq 200)
            {
            echo "[ OK ] CODE: $($_XML_CODE.Response.IPHost.Status.code) - Processed OK"
            $_LINEA_LOG | Out-File -Append -Encoding utf8 -FilePath "$($_LOGS_FOLDER)\OK.txt"
            } else {
            echo "[WARN] CODE: $($_XML_CODE.Response.IPHost.Status.code) - Error in execution $($_XML_CODE.Response.IPHost.Status.'#text')"
            $_LINEA_LOG | Out-File -Append -Encoding utf8 -FilePath "$($_LOGS_FOLDER)\NO_OK.txt"
            }
        }
        
    # For FQDN item
	# --------------
    if ($_OPERATION -eq 2)
        {
        if($_XML_CODE.Response.FQDNHost.Status.code -eq 200)
            {
            echo "[ OK ] CODE: $($_XML_CODE.Response.FQDNHost.Status.code) - Processed OK"
            $_LINEA_LOG | Out-File -Append -Encoding utf8 -FilePath "$($_LOGS_FOLDER)\OK.txt"
            } else {
            echo "[WARN] CODE: $($_XML_CODE.Response.FQDNHost.Status.code) - Error in execution $($_XML_CODE.Response.IPHost.Status.'#text')"
            $_LINEA_LOG | Out-File -Append -Encoding utf8 -FilePath "$($_LOGS_FOLDER)\NO_OK.txt"
            }
        }
	}



disclaimer
[edited by: FloSupport at 2:14 AM (GMT -8) on 26 Nov 2020]
Parents
  • Hi Antonio, thanks for sharing! Will have a look and provide feedback if needed. 

    Cheers

    Intrusus
    Sophos Certified Engineer | Sophos Certified Technician

    private lab:
    XG firewall with SFOS 18.0.3 MR-3
    Intercept X Advanced (for Server) with EDR EAP latest
    If a post solves your question use the 'Verify Answer' link

Reply
  • Hi Antonio, thanks for sharing! Will have a look and provide feedback if needed. 

    Cheers

    Intrusus
    Sophos Certified Engineer | Sophos Certified Technician

    private lab:
    XG firewall with SFOS 18.0.3 MR-3
    Intercept X Advanced (for Server) with EDR EAP latest
    If a post solves your question use the 'Verify Answer' link

Children
No Data