Important note about SSL VPN compatibility for 20.0 MR1 with EoL SFOS versions and UTM9 OS. Learn more in the release notes.
Hi Andrew,
Looks like you are close to a working solution. I too - just as you have tried - create a multipart/form-data object (System.Net.Http.MultipartFormDataContent).
My request looks something like the following (highlighted in yellow where I believe you need to adjust):
--6332ebe1-9b29-49f6-a4f2-1728b60e4131
Content-Type: application/xml; charset=utf-8
Content-Disposition: form-data; name=reqxml
<Request>
<Login>
<Username>*** USER ***</Username>
<Password passwordform="encrypt">*** HASHED PASSWORD - remove encrypt to use non-hashed ***</Password>
</Login>
<Set operation="add">
<Certificate>
<Name>*** CERT NAME ***</Name>
<Action>UploadCertificate</Action>
<CertificateFormat>pkcs12</CertificateFormat>
<Password>*** PASSWORD ***</Password>
<CertificateFile>*** CERT NAME ***.pfx</CertificateFile>
</Certificate>
</Set>
</Request>
--6332ebe1-9b29-49f6-a4f2-1728b60e4131
Content-Disposition: form-data; filename=*** CERT NAME ***.pfx; name=*** CERT NAME ***
Content-Type: application/octet-stream
*** BINARY DATA ***
--6332ebe1-9b29-49f6-a4f2-1728b60e4131--
With this approach I get the following response:
<?xml version="1.0" encoding="UTF-8"?>
<Response APIVersion="1700.1">
<Login>
<status>Authentication Successful</status>
</Login>
<Certificate transactionid="">
<Status code="200">Configuration applied successfully.</Status>
</Certificate>
</Response>
Also remember that you cannot upload a certificate with a name that already exists. You need to create a new unique name each time (then update all references to use the new and then delete the old).
A final note here is that it seems that firewall rules that have Path-specific routing looses some of its configuration on update (in my case the Allowed Client Networks). Haven't fully investigated what I'm doing wrong here, but to me it seems to be a bug in the API as I on update return the same configuration the API gives me...
Hope this helps!
Trond
That's great. Thanks Trond!
The missing link was putting the XML in the request body as well as the certificate, rather than just in the URL. All working now.
Thanks for the help. [:)]
Any chance that either of you could do a complete write-up on doing this. ( I would prefer something that runs from a UNIX host :) ) I am assuming that Lets Encrypt certificates are being generated using --check-dns so it would only require one central system to make the various requests for each site/service that would require a certificate.
What is the impact of having to add a new certificate and delete the old one for the selected sites/services.
This would be awesome if it all could be done in the WEB and Mail services.
What would be even better if Sophos would just include Lets Encrypt support in the XG directly and us not having to create these crazy work arounds.
Good work to you two guys.
-Ron
Andrew, Thank you for continuing the work and reporting your findings. Hopefully someone from Sophos will pick this up and enlighten us. :)
-Ron
I'm gonna kick this one once more. I'm also trying to update the certificate of my Sophos XG instance using a Powershell script. I've managed to get the multipart request exactly the same as in the above post. But for some reason Sophos XG only returns a 200 OK without any body. So there must be something wrong, but I've gone over the request body 10 times now an I'm fairly sure that its ok. So if someone could maybe share their powershell script for reference, that would be awesome.
Update: I managed to get to the point where it now responds (newlines in the XML are a no-no it seems). But the response is always 510 Deleting entity referred by another entity. Going to debug some more...
Hi Trond and Andrew,
Firstly, thanks for the detailed question, it has helped me get closer to uploading the cert. I think.
This is the PowerShell script that I have got so far (based a lot off this page: http://blog.majcica.com/2016/01/13/powershell-tips-and-tricks-multipartform-data-requests/)
$ContentType = "application/octet-stream"
$certFile = "C:\temp\LE_cert.pfx"
$fileName = Split-Path $certFile -leaf
$boundary = [guid]::NewGuid().ToString()
$fileBin = [System.IO.File]::ReadAllBytes($certFile)
$enc = [System.Text.Encoding]::GetEncoding("iso-8859-1")
$template = @'
--{0}
Content-Type: application/xml; charset=utf-8
Content-Disposition: form-data; name=reqxml
<Request>
<Login>
<Username>apiUSER</Username>
<Password>apiPWD</Password>
</Login>
<Set operation="add">
<Certificate>
<Name>LE_tcxapi_20181105_cert</Name>
<Action>UploadCertificate</Action>
<CertificateFormat>pkcs12</CertificateFormat>
<Password>pwd</Password>
<CertificateFile>{1}</CertificateFile>
</Certificate>
</Set>
</Request>
--{0}
Content-Disposition: form-data; filename="{1}"; name="Unique"
Content-Type: {2}
{3}
--{0}--
'@
$body = $template -f $boundary, $fileName, $ContentType, $enc.GetString($fileBin)
$response = Invoke-WebRequest -UseBasicParsing -Headers $headers -Uri 'fw-exg-00:4444/.../APIController operation="add"><Certificate><Name>MHA_TEST_Cert</Name><Action>UploadCertificate</Action><CertificateFormat>pkcs12</CertificateFormat><Password>pwd</Password><CertificateFile>LE_cert.pfx</CertificateFile></Certificate></Set></Request>' -Method POST -body $body
This gives me the following request body:
--db611ddb-ea8a-450f-aad8-90bee1c6a889
Content-Type: application/xml; charset=utf-8
Content-Disposition: form-data; name=reqxml
<Request>
<Login>
<Username>apiUSER</Username>
<Password>apiPWD</Password>
</Login>
<Set operation="add">
<Certificate>
<Name>LE_cert</Name>
<Action>UploadCertificate</Action>
<CertificateFormat>pkcs12</CertificateFormat>
<Password>password1</Password>
<CertificateFile>LE_cert.pfx</CertificateFile>
</Certificate>
</Set>
</Request>
--db611ddb-ea8a-450f-aad8-90bee1c6a889
Content-Disposition: form-data; filename="LE_cert.pfx"; name="LE_cert"
Content-Type: application/octet-stream
<<--redactedEncodedTXT-->>
--db611ddb-ea8a-450f-aad8-90bee1c6a889--
I think I have got my request in the same format as Trond's example but I am still getting "<Status code="500">Operation could not be performed on Entity.</Status>"
I was wondering how you were encoding your certificate, I have tried the above iso-8859-1, but also UTF-8 and Base64 and none of them have got the certificate uploaded.
I think I have got this right, but I just can't get the certificate uploaded, so any help would be greatly appreciated.
Cheers,
Tim
curl -k -F "reqxml=<uploadTest.txt" -F file=@Certificates.p12 "https://<DNS or IP here>:4444/webconsole/APIController?"
What mattered was using the @ and not < for the "file=" section.
From the Curl man page:
The difference between @ and < is then that @ makes a file get attached in the post as a file upload, while the < makes a text field and just get the contents for that text field from a file.
The rest of this post is for those still learning/trying to figure things out:
---Contents of the file called uploadTest.txt---
<Request>
<Login>
<Username>username</Username>
<Password>password</Password>
</Login>
<Set operation="add">
<Certificate>
<Action>UploadCertificate</Action>
<Name>friendly name for us humans goes here</Name>
<Password>password to decrypt the certificate file</Password>
<CertificateFormat>pkcs12</CertificateFormat>
<CertificateFile>Certificates.p12</CertificateFile>
<PrivateKeyFile></PrivateKeyFile>
</Certificate>
</Set>
</Request>
---End content of file uploadTest.txt---
**In the example above, Certificates.p12 is a placeholder for the filename of your certificate, and if you don't have a separate private key file you can omit the PrivateKeyFile line.
I created a .p12 file for my certificate and private key simply because it was less to type in the file above and command line when testing. Valid formats are listed in the API documentation
Most likely, if you have trouble in uploading the certificate (any file to XG via API), use a Test page and check, what you "actually" upload to the server / xg.
http://ptsv2.com/s/howitworks.html
Most likely your file is not proper uploaded / wrong format etc.
So you should verify, what you are uploading because XG has no proper storage of the file, if the request was invalid. It will simply flush the upload after parsing the command.
__________________________________________________________________________________________________________________
Thanks everyone who contributed to this thread. It has been valuable to me figuring out how to upload the Let's Encrypt certificates from my Synology NAS to Sophos XG. Here's how it is done.
Enable API (optionally create a special API Administration user) as described here: https://community.sophos.com/kb/en-us/132560
On your Synology NASfollow the instructions for Let's Encrypt here and include your firewall's fqdn as a subject alternative name: https://www.synology.com/en-global/knowledgebase/DSM/help/DSM/AdminCenter/connection_certificate
Then create this XML file, e.g. in your home directory:
<?xml version="1.0" encoding="UTF-8"?>
<Request APIVersion="1702.1">
<!-- API Authentication -->
<Login>
<Username>apiuser</Username>
<Password>randompw</Password>
</Login>
<Set operation="add">
<Certificate>
<Action>UploadCertificate</Action>
<Name>yourdomain</Name>
<CertificateFormat>pem</CertificateFormat>
<CertificateFile>yourdomain.pem</CertificateFile>
<PrivateKeyFile>yourdomain.key</PrivateKeyFile>
</Certificate>
</Set>
</Request>
Under Control Panel, Task Scheduler, create the following User-defined script as Scheduled Task, that runs as User root.
/bin/curl -F "reqxml=</var/services/homes/youruser/updatecertificate.xml" -F "file=@/usr/syno/etc/certificate/system/default/cert.pem;filename=yourdomain.pem" -F "file=@/usr/syno/etc/certificate/system/default/privkey.pem;filename=yourdomain.key" -k https://yourfirewall:4443/webconsole/APIController
Click "Run" to test and run it once. You should now have your Synology certificate and private key under SYSTEM, Certificates. If that worked, then make the following change in the the XML file: <Set operation="update">
That should be it. From now on your firewall should be certified by Let's Encrypt and updated timely with renewed certificates. I run the task weekly on Sunday morning.
Cheers!
I have finally got this setup. The add worked properly but when the update executes every Sunday I get this output
<?xml version="1.0" encoding="UTF-8"?>
<Response APIVersion="1702.1" IPS_CAT_VER="1">
<Login>
<status>Authentication Successful</status>
</Login>
<Certificate transactionid="">
<Status code="500">Operation could not be performed on Entity.</Status>
</Certificate>
</Response>
Any ideas on what I am doing wrong. I have confirmed that I have changed the operation from add to update.
So basically you can add with your script a Certificate, but you cannot update it?
As far as i know, you cannot "overwrite" the used Certificate. Because it is loaded in different places by XG.
What you have to do, would be reupload the certificate with different namens and change those uses in each place, you like.
__________________________________________________________________________________________________________________
The 'Set operation="update"' works fine for me. This is the output that I see at the weekly update:
<?xml version="1.0" encoding="UTF-8"?>
<Response APIVersion="1702.1" IPS_CAT_VER="1">
<Login>
<status>Authentication Successful</status>
</Login>
<Certificate transactionid="">
<Status code="200">Configuration applied successfully.</Status>
</Certificate>
</Response>
Can you elaborate on the permission issue? I'm having the same problem