Thread Info
  • State Not Answered
  • Locked Locked
  • Replies 1 reply
  • Subscribers 38 subscribers
  • Views 7050 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Letsencrypt API Update Script - dynamically handles multiple certs, multiple rules, including re-grouping of policies rules

burton

I wanted a way to auto update my letsencrypt certificates for use on my XG firewall and WAF rules. I developed this script to handle multiple certificates, and to be as dynamic as possible. The approach I took to achieve this is the following:

1) Within the script declare the single or multiple certificates you want to update on your xg firewall ( all the certs to be updated must be locally accessible )

2) the script uses the api to discover all firewall rules that are currently in use by any of the declared certs

3) it then discovers all firewall groups that any of the discovered rules are apart of ( an api update to a rule, removes it from it's firewall group )

4) it then creates a temporary certificate so that the certificate needing updated can be replaced/updated without error

5) it then applies the temporary certificate to any of the rules that need to have their cert updated ( thus allowing the update of the cert without error )

6) it then uploads all declared updated certificates

7) it then goes back through every rule using each certificate and re-assigns them to the original certificate, which is now updated

8) finally it goes through and re-assigns all updated rules back into their assigned firewall groups ( thus leaving everything as it was before, only now with the updated certs )

I have tested this script in my environment with no issues, and two other xg firewall environment with no issues. Please use at your own risk, and be sure to have a backup of you xg config before attempting to use this! I hope this is helpful for others who are wanting to solve this same issue. Thanks to all the other posts about this in the sophos community. They were a big help to kickstart me on this project!

This attached script is written in php and utilized php curl to communicate to the xg api. It is also meant to be run from the command line of a linux o/s. It can also be setup to be called via certbot with the following line "certbot renew --post-hook /your/path/here/certbot_posthook.sh" where certbot_posthook.sh contains:

#!/bin/sh
php /your/path/here/update_xg_cert.php

adding --dry-run to certbot can be used to test, otherwise certbot will only call post-hook when a cert is updated

certbot renew --dry-run --post-hook /your/path/here/certbot_posthook.sh

The following is the php code for the "update_xg_cert.php" file. Be sure to go through the "SET VARIABLES" section of the code and declare all the settings to match your environment. The default examples are setup to demo multiple certificates. If you only have one cert, then remove the "d.e.f" examples. Leaving only one value in the arrays. ie:

$xg_certificate_files = array("fullchain.pem"); //names of your cert files to be used in xg

$xg_certificate_keys = array("privkey.key");

$certificate_file_uploads = array("/etc/letsencrypt/live/a.b.c/fullchain.pem");

$certificate_key_uploads = array("/etc/letsencrypt/live/a.b.c/privkey.pem");


Or if you have 3 or more, just continue to add values to the arrays for each certificate

*** NOTE *** This is written to use the PEM certificate format, so you must provide a certificate.pem file and a privatekey.pem file to this script. 

You can then run it by typing "php ./update_xg_cert.php"

<?php
/*
print "\e[35mPURPLE\e[39m\n";
print "\e[32mGREEN\e[39m \n";
print "\e[31mRED\e[39m \n";
*/
//================ START SET VARIABLES ===================
$xg_ip = ""; //ip of your xg firewall for api access
$username = ""; //api username for xg firewall with api access
$password = ""; // api username password
$temp_cert = "DO_NOT_REMOVE_-_CERTBOT_TEMP";

$update_certs = array("a.b.c","d.e.f"); //names of the certificates on XG that you are wanting to update
$xg_certificate_files = array("fullchain.pem","fullchain.pem"); //names of your cert files to be used in xg
$xg_certificate_keys = array("privkey.key","privkey.key"); // name of your cert key files to be used in xg  ( note it must end in .key )
$certificate_file_uploads = array("/etc/letsencrypt/live/a.b.c/fullchain.pem","/etc/letsencrypt/live/d.e.f/fullchain.pem"); //local locations of cert file to upload
$certificate_key_uploads = array("/etc/letsencrypt/live/a.b.c/privkey.pem","/etc/letsencrypt/live/d.e.f/privkey.pem"); //local locations of key file to upload ( filename does not have to be .key but the filename above does have to be )
$cert_uploads = null;
$restore_policies = null;
$rule_names = array();
$fw_groups_to_restore = null;
//================ END SET VARIABLES ========================


//================ START CHECK FILES ===================
foreach($certificate_file_uploads as $key => $filename)
{
        if(!file_exists($filename))
        {
                print "\e[31mFILE NOT FOUND OR NOT READABLE BY CURRENT USER - CAN NOT CONTINUE: $filename\n\e[39m";
                die();
        }
}
foreach($certificate_key_uploads as $key => $filename)
{
        if(!file_exists($filename))
        {
                print "\e[31mFILE NOT FOUND OR NOT READABLE BY CURRENT USER - CAN NOT CONTINUE: $filename\n\e[39m";
                die();
        }
}
//================ END CHECK FILES ======================== 




function xml_curl($xg_ip,$username,$password,$xml,$cert_uploads)
{
	$inputxml =  '<?xml version="1.0" encoding="UTF-8"?><Request APIVersion="1800.1"><!-- API Authentication --><Login><Username>'.$username.'</Username><Password>'.$password.'</Password></Login>';
	$inputxml .= $xml;
	$inputxml .= "</Request>";


	$url="https://$xg_ip:4444/webconsole/APIController";

	$ch = curl_init();
	if (!$ch) {
	    die("Couldn't initialize a cURL handle");
	}
	
	
	$args = array();
	unset($args);
	curl_setopt($ch, CURLOPT_URL, $url);
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
	curl_setopt($ch, CURLOPT_TIMEOUT, 60);
	curl_setopt($ch, CURLOPT_POST, true);
	if($cert_uploads)
	{
		$args["reqxml"] = $inputxml;
		$args["file"] = curl_file_create($cert_uploads["certificate_file_upload"],"application/x-pem-file",$cert_uploads["xg_certificate_file"]);
		$args["file2"] = curl_file_create($cert_uploads["certificate_key_upload"],"application/x-pem-file",$cert_uploads["xg_certificate_key"]);
	}
	else
		$args["reqxml"] = $inputxml;
	
	curl_setopt($ch, CURLOPT_POSTFIELDS, $args); 
	curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
	curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
	curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
	$result = curl_exec($ch); // execute
	//var_dump($result);
	//$info = curl_getinfo($ch);
	//print_r($info);
	curl_close($ch);
	return($result);
}




//============= START OF CREATING TEMP CERT ====================
$create_temp_cert = '
<Set operation="add">
	<Certificate>
		<Action>GenerateSelfSignedCertificate</Action>
		<Name>'.$temp_cert.'</Name>
		<Password>Password</Password>
		<ValidFrom>2021-02-03</ValidFrom>
		<ValidUpto>2036-02-03</ValidUpto>
		<KeyType>RSA</KeyType>
		<KeyLength>2048</KeyLength>
		<SecureHash>SHA - 256</SecureHash>
		<KeyEncryption>Disable</KeyEncryption>
		<CertificateIDType>Email</CertificateIDType>
		<CertificateID>na@na.net</CertificateID>
		<CountryName>US</CountryName>
		<StateProvinceName>NA</StateProvinceName>
		<LocalityName>NA</LocalityName>
		<OrganizationName>NA</OrganizationName>
		<OrganizationUnitName>NA</OrganizationUnitName>
		<CommonName>na.na.na</CommonName>
		<EmailAddress>na@na.na</EmailAddress>
	</Certificate>
</Set>';

print "\n \e[35mCREATING TEMP CERT...\e[39m\n";
$xmlstring = xml_curl($xg_ip,$username,$password,$create_temp_cert,$cert_uploads);
$xg_result = new SimpleXMLElement($xmlstring);
$result = $xg_result->asXML();
if(preg_match("/successfully/i",$result))
	print "\e[32m$result\e[39m";
else
{
        print "\e[31m".$create_temp_cert."\n$result\e[39m";
        die("FAILED TO CREATE TEMP CERT - CAN NOT CONTINUE!\n");
}
//============= END OF CREATING TEMP CERT ====================   




foreach($update_certs as $key => $update_cert)
{
	unset($restore_policies);
	print "\n\n\n";

	$xg_certificate_file = $xg_certificate_files[$key];
	$xg_certificate_key = $xg_certificate_keys[$key];
	$certificate_file_upload = $certificate_file_uploads[$key];
	$certificate_key_upload = $certificate_key_uploads[$key];


	//===================== START FINDING FIREWALL POLICIES/RULES WITH THE CERT ASSIGNED =================

	$getprofiles = "<Get><FirewallRule></FirewallRule></Get>";
	$xmlstring = xml_curl($xg_ip,$username,$password,$getprofiles,$cert_uploads);


	$xml = new SimpleXMLElement($xmlstring);

	//print $xml->asXML();

	unset($rule_names);
	print "\e[35mFINDING POLICIES USING CERT: $update_cert\e[39m\n";
	foreach($xml->FirewallRule as $policy)
	{
		if(isset($policy->HTTPBasedPolicy[0]->Certificate[0]))
		{
			if($policy->HTTPBasedPolicy[0]->Certificate[0] == $update_cert)
			{
				print "FOUND MATCHING POLICY RULE TO BE UPDATED: ".$policy->Name[0]."\n";
				$rule_names["rule_name"][] = (string)$policy->Name[0];
			}
		}
	}
	//print $xml->asXML();



	//FIND FIREWALL GROUPS THAT THE FOUND RULE MAY BE APART OF TO RESTORE LATER ( updating the cert via api, removes the rule from the group )
	$getfwgroups = "<Get><FirewallRuleGroup></FirewallRuleGroup></Get>";
	$xmlstring = xml_curl($xg_ip,$username,$password,$getfwgroups,$cert_uploads);
	$fwgroup_xml = new SimpleXMLElement($xmlstring);
	//print $xml->asXML();
	foreach($rule_names["rule_name"] as $rule_key => $rule_name)
	{
	
		if(isset($fwgroup_xml->FirewallRuleGroup))
		{
			print "\e[35mFINDING ASSIGNED GROUPS FOR RULE: $rule_name\e[39m\n";
			foreach($fwgroup_xml->FirewallRuleGroup as $fwgroup)
			{
				if(isset($fwgroup->SecurityPolicyList[0]))
				{
					//print "debug: ".$fwgroup->SecurityPolicyList->asXML();
					if(isset($fwgroup->SecurityPolicyList[0]->SecurityPolicy[0]))
					{
						foreach($fwgroup->SecurityPolicyList[0]->SecurityPolicy as $security_policy)
						{
							//print "dapolicy: ".$security_policy->asXML();
							if($security_policy[0] == $rule_name)
							{
								print "FOUND MATCHING FIREWALL RULE GROUP: ".$fwgroup->Name[0]."\n";
								if(!isset($fw_groups_to_restore["names"][(string)$fwgroup->Name[0]])) //we only set this key once, because if the next cert is in the same group, then it would update it with the last cert missing
									$fw_groups_to_restore["names"][(string)$fwgroup->Name[0]] = $fwgroup->asXML();
							}
						}
					}
				}
			}
		}	
	}



	//SETTING TEMP CERT
	print "\n\n\n";
	foreach($xml->FirewallRule as $policy)
	{
		if(isset($policy->HTTPBasedPolicy[0]->Certificate[0]))
		{
			if($policy->HTTPBasedPolicy[0]->Certificate[0] == $update_cert)
			{
				$restore_policies[] = $policy->asXML();

				print "\e[35mAPPLYING TEMP CERT FOR POLICY RULE: ".$policy->Name[0]."\e[39m\n";
				$policy->HTTPBasedPolicy[0]->Certificate[0] = $temp_cert;
				

				$update_policy = '<Set operation="update">';
				$update_policy .= $policy->asXML();
				$update_policy .= '</Set>';
				//print $update_policy."\n";
				$xmlstring = xml_curl($xg_ip,$username,$password,$update_policy,$cert_uploads);
				$xg_result = new SimpleXMLElement($xmlstring);
				//print $xg_result->asXML();
				$result = $xg_result->asXML();
				if(preg_match("/successfully/i",$result))
					print "\e[32m$result\e[39m";
				else
					print "\e[31m".$update_policy."\n$result\e[39m";
				print "\n\n";
			}
		}
	}
	//=============== END FINDING OF THE FIREWALL POLICIES/RULES WITH THE CERT ASSIGNED ======================



	//================ START UPLOAD/UPDATE OF LETSENCRYPT CERT ==================
	print "\n\n\n";
	print "\e[35mUPLOADING UPDATED CERTs\e[39m\n\n";
	$update_cert_xml = '<Set operation="update">
		<Certificate>
			<Action>UploadCertificate</Action>
			<Name>'.$update_cert.'</Name>
			<CertificateFormat>pem</CertificateFormat>
			<CertificateFile>'.$xg_certificate_file.'</CertificateFile>
			<PrivateKeyFile>'.$xg_certificate_key.'</PrivateKeyFile>
		</Certificate>
	</Set>  ';

	$cert_uploads["certificate_file_upload"] = $certificate_file_upload;
	$cert_uploads["xg_certificate_file"] = $xg_certificate_file;
	$cert_uploads["certificate_key_upload"] = $certificate_key_upload;
	$cert_uploads["xg_certificate_key"] = $xg_certificate_key;


	$xmlstring = xml_curl($xg_ip,$username,$password,$update_cert_xml,$cert_uploads);
	unset($cert_uploads);
	$xg_result = new SimpleXMLElement($xmlstring);
	//print $xg_result->asXML();
	$result = $xg_result->asXML();
	if(preg_match("/successfully/i",$result))
		print "\e[32m$result\e[39m";
	else
		print "\e[31m".$update_policy."\n$result\e[39m";
	print "\n\n\n";

	//unset($cert_uploads);
	$cert_uploads = null;
	//=========== END UPDATE OF LETSENCRYPT CERT ======================






	//========== START OF PUTTING THE UPDATED FIREWALL POLICIES/RULES BACK TO ORIGINAL UPDATED CERT ===========
	foreach($restore_policies as $key => $restore_xml)
	{
		$policy = new SimpleXMLElement($restore_xml);
		//print $policy->asXML();
		print "\e[35mAPPLYING UPDATED ORIGINAL CERT FOR POLICY RULE: ".$policy->Name[0]."\e[39m\n";

		$update_policy = '<Set operation="update">';
		$update_policy .= $restore_xml;
		$update_policy .= '</Set>';

		$xmlstring = xml_curl($xg_ip,$username,$password,$update_policy,$cert_uploads);
		$xg_result = new SimpleXMLElement($xmlstring);
		//print $xg_result->asXML();
		$result = $xg_result->asXML();
		if(preg_match("/successfully/i",$result))
			print "\e[32m$result\e[39m";
		else
			print "\e[31m".$update_policy."\n$result\e[39m";
		print "\n";
	}
	//========== END OF PUTTING THE UPDATED FIREWALL POLICIES/RULES BACK TO ORIGINAL UPDATED CERT ===========






} //end foreach loop to go through each cert



//============= START OF DELETING TEMP CERT ====================
print "\n\n\n";
$delete_temp_cert = '
<Remove>
	<Certificate>
		<Action>GenerateSelfSignedCertificate</Action>
		<Name>'.$temp_cert.'</Name>
		<Password>Password</Password>
		<ValidFrom>2021-02-03</ValidFrom>
		<ValidUpto>2036-02-03</ValidUpto>
		<KeyType>RSA</KeyType>
		<KeyLength>2048</KeyLength>
		<SecureHash>SHA - 256</SecureHash>
		<KeyEncryption>Disable</KeyEncryption>
		<CertificateIDType>Email</CertificateIDType>
		<CertificateID>na@na.net</CertificateID>
		<CountryName>US</CountryName>
		<StateProvinceName>NA</StateProvinceName>
		<LocalityName>NA</LocalityName>
		<OrganizationName>NA</OrganizationName>
		<OrganizationUnitName>NA</OrganizationUnitName>
		<CommonName>na.na.na</CommonName>
		<EmailAddress>na@na.na</EmailAddress>
	</Certificate>
</Remove>';

print "\n\n\n";
print "\e[35mDELETING TEMP CERT...\e[39m\n";
$xmlstring = xml_curl($xg_ip,$username,$password,$delete_temp_cert,$cert_uploads);
$xg_result = new SimpleXMLElement($xmlstring);
//print $xg_result->asXML();
$result = $xg_result->asXML();
if(preg_match("/successfully/i",$result))
	print "\e[32m$result\e[39m";
else
	print "\e[31m".$delete_temp_cert."\n$result\e[39m";
print "\n\n";
//============= END OF DELETING TEMP CERT ====================



//============= START OF RESTORING FIREWALL RULE GROUPS ====================
//print_r($fw_groups_to_restore);
if($fw_groups_to_restore)
{
	foreach($fw_groups_to_restore["names"] as $key => $fwgroup_xml)
	{
		$fwgroup = new SimpleXMLElement($fwgroup_xml);
		//print $policy->asXML();
		print "\e[35mRESTORING FIREWALL GROUP SETTINGS FOR: ".$fwgroup->Name[0]."\e[39m\n";

		$update_fwgroup = '<Set>';
		$update_fwgroup .= $fwgroup_xml;
		$update_fwgroup .= '</Set>';

		//print "update_fw_group: $update_fwgroup\n";

		$xmlstring = xml_curl($xg_ip,$username,$password,$update_fwgroup,$cert_uploads);
		$xg_result = new SimpleXMLElement($xmlstring);
		//print $xg_result->asXML();
		$result = $xg_result->asXML();
		if(preg_match("/successfully/i",$result))
			print "\e[32m$result\e[39m";
		else
			print "\e[31m".$update_policy."\n$result\e[39m";
		print "\n";

	}
}
//============= END OF RESTORING FIREWALL RULE GROUPS ====================

?>



This thread was automatically locked due to age.
Unfiltered HTML