API import successful - but failed

I was importing a firewall rule on a remote XGS and used a XML file for that.

While the GUI showed green message after importing .tar file "API import successful" the rule got not created.

The apiparser.log shows issues with the XML structure.

INFO      Aug 10 09:49:06Z [28730]: Tar File : /sdisk/API-1660124945370225.tar is created.INFO      Aug 10 09:55:36Z [3748]: Sanity check not required. And XML file is valid. xml: /sdisk/api-2022-08-10-11-55-36/Entities.xml.
INFO      Aug 10 09:55:36Z [3748]: Start Set Handler,Component : FirewallRule
ERROR     Aug 10 09:55:36Z [3748]: Key:ISCrEntity is not found in RequestMap File for FirewallRule.
WARNING   Aug 10 09:55:36Z [3748]: Can't get the <Add/Update> element from map file, So Mode value is 'Add'.
ERROR     Aug 10 09:55:36Z [3748]: Parser Error: xmlvalue for jsonkey="tempsourceid", xmlelement="/FirewallRule/NetworkPolicy/SourceNetworks/Network" cannot be found in request file.
ERROR     Aug 10 09:55:36Z [3748]: Parser Error: xmlvalue for jsonkey="srczones_exception", xmlelement="/FirewallRule/NetworkPolicy/Exclusions/SourceZones/Zone" cannot be found in request file.
ERROR     Aug 10 09:55:36Z [3748]: Parser Error: xmlvalue for jsonkey="dstzones_exception", xmlelement="/FirewallRule/NetworkPolicy/Exclusions/DestinationZones/Zone" cannot be found in request file.
ERROR     Aug 10 09:55:36Z [3748]: Parser Error: xmlvalue for jsonkey="sourceid_exception", xmlelement="/FirewallRule/NetworkPolicy/Exclusions/SourceNetworks/Network" cannot be found in request file.
ERROR     Aug 10 09:55:36Z [3748]: Parser Error: xmlvalue for jsonkey="serviceid_exception", xmlelement="/FirewallRule/NetworkPolicy/Exclusions/Services/Service" cannot be found in request file.
ERROR     Aug 10 09:55:36Z [3748]: Parser Error: xmlvalue for jsonkey="destinationid_exception", xmlelement="/FirewallRule/NetworkPolicy/Exclusions/DestinationNetworks/Network" cannot be found in request file.
ERROR     Aug 10 09:55:36Z [3748]: json object not found with key="tempsourceid" to handle logicaloperator.
ERROR     Aug 10 09:55:36Z [3748]: Parser Error: xmlvalue for jsonkey="sourceid", xmlelement="/FirewallRule/NetworkPolicy/SourceNetworks/Network" cannot be found in request file.
ERROR     Aug 10 09:55:36Z [3748]: type != const in logicaloperator.So string comparision is done.
ERROR     Aug 10 09:55:36Z [3748]: type != const in logicaloperator.So string comparision is done.
ERROR     Aug 10 09:55:37Z [3748]: Flag setting for this opcode is 16.
INFO      Aug 10 09:55:37Z [3748]: Opcode response: status:500

Is a green success on the GUI OK when it actually fails?



Edited TAGs
[edited by: emmosophos at 10:03 PM (GMT -7) on 16 Aug 2022]
Parents
  • Essentially there is a different in Import and failing of objects. This is currently a known issue, if you have multiple objects, which are passing but one is failing, the API import will stop but the "import of the file was successful". 

    Essentially this part / component of SFOS needs a rework to reflect this behavior in a better manner. 

    The problem is: If you import a file with ~1k objects, those object will be imported in the background. And this can take a while.

    But the import essentially (of the file .tar) was successful at this point. So we could stuck the webadmin for some time and let the admin wait until everything is imported, but that could lead to different issues, as we cannot freeze the webadmin for so long. So SFOS is taking the shortcut and show, the import of the file was successful but the object import is in the background. 

    Most likely customers will notice a failure, if a object is missing. 

    This is the downside of not using a API and a import of a file instead, as there is no "backchannel to tell about the failure". You would have to create a new format in the logviewer or something to inform about this failure. And this is something on the backlog. 

    __________________________________________________________________________________________________________________

Reply
  • Essentially there is a different in Import and failing of objects. This is currently a known issue, if you have multiple objects, which are passing but one is failing, the API import will stop but the "import of the file was successful". 

    Essentially this part / component of SFOS needs a rework to reflect this behavior in a better manner. 

    The problem is: If you import a file with ~1k objects, those object will be imported in the background. And this can take a while.

    But the import essentially (of the file .tar) was successful at this point. So we could stuck the webadmin for some time and let the admin wait until everything is imported, but that could lead to different issues, as we cannot freeze the webadmin for so long. So SFOS is taking the shortcut and show, the import of the file was successful but the object import is in the background. 

    Most likely customers will notice a failure, if a object is missing. 

    This is the downside of not using a API and a import of a file instead, as there is no "backchannel to tell about the failure". You would have to create a new format in the logviewer or something to inform about this failure. And this is something on the backlog. 

    __________________________________________________________________________________________________________________

Children
  • thanks for your detailled answer. Good to read, that Sophos has this on the 2do list. I would not want the webadmin to be unesponsive for half an hour when importing such a huge list.

    in our case here it was only a single firewall rule but with all the 300+ Zoom network objects. that should go quickly, not even 5 minutes.

    the zoom network objects (hopefully) all already exist as they have been imported before but probably this is the thing that you meat with "Most likely customers will notice a failure, if a object is missing.". Actually it is now more work to filter out if and what is missing here.