FileMaker does not accept reverse proxy SSL certificate

Hi people,

I hope this is the correct forum. It took me some time before I found out how to even post a question.

I have this FileMaker database server which is sitting in the private network. Using NAT, I have configured all traffic over port 5003 to go to the internal IP of that server.

Because it is using SSL encryption, I have set up a reverse proxy entry for https, and linked my domain it's wild card certificate to it, and I am passing host headers.

That worked fine until some months ago, don't know exactly, probably after some upgrade or update.

Now, when I connect from the outside to the server, it is telling me that the certificate is invalid. There is a button to look at the certificate from that dialog, and it shows that all parts of the certificate ( cert, intermediate, CA ) are all OK.

This does not happen when I connect from the inside, using the same DNS name.

I realise this is not exactly main stream http / https proxying, but just maybe someone has experience here with this particular setup.

  • I am confused.   If you use a reverse proxy (WAF), you do not use NAT.

    Are you accessing FileMaker with a browser or a custom application?

    UTM WAF has an unwanted "feature" that causes it to include the root certificate in the certificate chain.  This is technically incorrect, because the root certificate is useless it is pre-installed on the client device, and as a self-signed certificate in the certificate chain should be untrusted by the receiving system.   Browsers will ignore the unwanted certificate, but some custom software might not.   Check to see if you need to configure a certificate-checking override.

     

  • In reply to DouglasFoster:

    Hi Douglas,

    DouglasFoster
    I am confused.   If you use a reverse proxy (WAF), you do not use NAT.

    The WAF is for the https part, the NAT is for the database connection over TCP 5003. Both are needed. The WAF part takes part of the initial connection and authentication, and later on for blob/container streaming.

    DouglasFoster
    Are you accessing FileMaker with a browser or a custom application?

    I can use both the web browser and the regular FileMaker client. The web browser doesn't give any problems, it's the FileMaker client that is being difficult about the certificate. I have an additional SSL web site running using the web application firewall, using the same wildcard certificate, and it is running fine.

    DouglasFoster
    UTM WAF has an unwanted "feature" that causes it to include the root certificate in the certificate chain.  This is technically incorrect, because the root certificate is useless it is pre-installed on the client device, and as a self-signed certificate in the certificate chain should be untrusted by the receiving system.   Browsers will ignore the unwanted certificate, but some custom software might not.

    Let me try to rephrase that, so you can check if I understand it. The Sophos UTM is relaying ALL the certificate information, the certificate itself, the intermediate one, AND the root certificate, and this last one should not be sent. The FileMaker client ( custom software ) is not accepting that, the check button is just using the WebKit or IIS browser frameworks to verify the certificate and sees no problem at all. This seems to fit what I am experiencing.

    DouglasFoster
    Check to see if you need to configure a certificate-checking override.

    I do not know how to configure an override, I would like to test with some other certificate that does not include the CA. Is it possible to upload that using the Sophos web console? Or do I have to connect using SSH and configure this from the terminal?

    Thanks for your help so far.

  • In reply to clarifix:

    The problem may be related to the dual sessions.    The WAF accepts a connection from you and creates a connection, on your behalf, to the web site.   The remote system is probably having difficulty matching the two sessions.   There is an X-Forwarded-For header that UTM and other WAF devices add to the relay session so that the receiving webserver has the option of seeing the source address instead of the UTM address as the client.

    I think your next inquire should be to your FileMaker support channels to see how / whether it works when a WAF device is involved.

  • In reply to DouglasFoster:

    DouglasFoster
    I think your next inquire should be to your FileMaker support channels to see how / whether it works when a WAF device is involved.

    I fear that is not a good option.

    FileMaker support is not knowledgeable enough to answer questions like these. Maybe after a lot of explaining they will understand the question, but when/if they would, I am 99% sure they would respond that this is not not supported. Or something like that. Setups like this are impossible for them to reproduce, they would need a networking lab, so I think FileMaker support will not provide answers.

    I wil have more luck I guess with the US FileMaker forums. I will ask my question there.

    Thanks Douglas.

  • In reply to clarifix:

    You may be able to figure it out on your own.  The first step is to see if anything is being blocked.   I am comparing your question to this situation:

    Some remote access methods use two sessions on two different ports.   I will use Citrix as an example:

    • On a web page, you log into Citrix, obtain a menu of possible sessions, and click one of them to launch it.   
    • Citrix sends back a launch.ica file which contains the name and port of the server to use, a code to represent the session to launch, and a code to represent your user login session.
    • The Citrix plug-in for your web browser interprets the launch.ica file and makes the connection using Citrix proprietary protocols.

    For Citrix to work with UTM, the web session can go through the WAF, but the second session has to be allowed through the firewall rules.

    For your situation:

    • FileMaker may see the WAF session and the database connection as coming from two different addresses, and reject it on that basis.
    • The secret code or other information that represents your login session may get lost in the handoff from WAF to client.
    • The second session may be non-HTML traffic sent to the WAF address and port, where UTM will fail it because WAF can only handle HTML.

    If you are the only user, or one of a very small group of users, then Remote Access over SSL VPN should solve all these problems and provide equivalent or greater security, without WAF.

    Unsolicited advice:

    Always use 2-factor authentication with remote access.  It is required by PCI DSS standards, might be needed for GDPR compliance, and is just good practice.  There are a lot of bad guys doing password guessing attacks on a lot of Internet-facing resources.

  • Hi,

    I'm currently trialling FileMaker and hit this problem (for some reason I'm now struggling with the 5003 NAT even that 443 with WAF is fine).

    Anyway, as has already been mentioned in other posts, the reverse proxy adds in extra headers (x-forwarded-****).

    This appears to mess with the URL Rewrite rules in IIS that FileMaker uses to point public addresses to local services.

    The fix for me was for add in my own rewrite rule in the site web.config file (at C:\Program Files\FileMaker\FileMaker Server\HTTPServer\conf in my installation).

    I put it as the first rule, to remove the offending headers before FileMaker deals with the request.

    <rule name="X-forward" patternSyntax="Wildcard">
        <match url="*" />
        <conditions logicalGrouping="MatchAll" trackAllCaptures="false" />
        <serverVariables>
            <set name="HTTP_X_FORWARDED_FOR" value="" />
            <set name="HTTP_X_FORWARDED_SERVER" value="" />
            <set name="HTTP_X_FORWARDED_HOST" value="" />
        </serverVariables>
        <action type="None" />
    </rule>

    I had to add in HTTP_X_FORWARDED_FOR and HTTP_X_FORWARDED_SERVER as additional Server Variables in IIS Manager (HTTP_X_FORWARDED_HOST was already allowed by default).

  • In reply to gilesmhb:

    Hi Giles,

    Thanks very much for sharing this.

    I followed your instructions and added the rule add the bottom of the rules section in the config XML file.

    I also added the 2 server variables in IIS, at the level of the FMS site ( where they are local ). IIS then wrote some extra lines in the config file:

    <allowedServerVariables>
    <add name="HTTP_X_FORWARDED_SERVER" />
    <add name="HTTP_X_FORWARDED_FOR" />
    </allowedServerVariables>

    I rebooted the OS just to make sure and tried it out from an internet machine, but it didn't solve the problem.

    But I think I know now what the problem is. See the next post.

  • I used the excellent tool on SSLlabs.com ( https://www.ssllabs.com/ssltest ) and discovered that the intermediate certificate is incomplete.

    So I think the WAF handles the main (wildcard) certificate, but not the intermediate one.

    Which appears logical to me, because the WAF web interface only lets me configure the main certificate.

    Could that be the reason?

    If that is the reason, could there be some way to make a certifcate file for the firewall that includes both the main certificate and the intermediate one?

  • I found the solution to the problem. Using www.markbrilman.nl/.../ as documentation I created a pfx file that contains the main AND the intermediate certificate. I first removed the old wildcard crt file from the firewall, then imported the pfx file, and assigned it to the virtual servers that run over https. They now return a green A sign on sslabs and... the FileMaker client problem disappeared!! Thanks for helping me understand and fix this problem.