Solution: Configure to NOT block Spotify in Web Filtering

This present a methodology for identifying what's necessary for unblocking access to a site.  It's applicable to figuring out how to unblock sites other than Spotify's..

One of my clients is a very successful estate and tax attorney.  Last month, we added an SG 105w to his office.  When the basic configuration blocked Spotify, I ran the following commands (my bold) and made the configurations in the picture.  Note that "416" and "429" are only resolvable by skipping the Proxy.  "504" also requires skipping if an Exception for AV scanning doesn't resolve it. 

I would be glad to get suggestions for improvement of this post.  In specific, if someone plays with this to determine the minimum things to skip in the Exception, that would be cool!

Cheers - Bob

secure:/root # grep spotify /var/log/http.log|grep -oP 'url="https://.*?\/'|sort -n|uniq -c
         2 url="https://api.spotify.com/
       10 url="https://apresolve.spotify.com/
         6 url="https://cdn.cookielaw.org/
         1 url="https://exp.wg.spotify.com/
         2 url="geolocation.onetrust.com/
         1 url="gew-dealer.spotify.com/
         1 url="gew-spclient.spotify.com/
         13 url="guc-dealer.spotify.com/
         5 url="guc-spclient.spotify.com/
         6 url="https://i.scdn.co/
       91 url="https://open.scdn.co/
       20 url="https://open.spotify.com/
         1 url="https://pixel.spotify.com/
         1 url="pixel-static.spotify.com/
         18 url="https://sentry.io/
         3 url="sp-bootstrap.spotifycdn.com/
         4 url="spclient.wg.spotify.com/
         2 url="https://unpkg.com/
         1 url="https://www.google.com/
       41 url="https://www.scdn.co/
       23 url="https://www.spotify.com/
secure:/root # grep spotify /var/log/http.log|grep -oP 'statuscode=".*?"'|sort -n|uniq -c
         8 statuscode="101"
     259 statuscode="200"
         3 statuscode="301"
         2 statuscode="302"
         7 statuscode="304"
         5 statuscode="416"
         3 statuscode="429"
         5 statuscode="504"
secure:/root # grep spotify /var/log/http.log|grep 'statuscode="416"'|tail -1
2021:04:26-14:27:36 secure httpproxy[5914]: id="0002" severity="info" sys="SecureWeb" sub="http" name="web request blocked" action="block" method="GET" srcip="192.168.1.142" dstip="23.47.193.33" user="" group="" ad_domain="" statuscode="416" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="0" request="0xcd8b7500" url="https://audio-akp-quic-spotify-com.akamaized.net/audio/e35adf4f62d790f07913c2a110554d73f446e53d?__token__=exp=1619551656~hmac=f1bd5f1e43c29d3918d4aef64a373ca1c44cc11f317b431db208c59075a5b5f3" referer="">https://open.spotify.com/" error="" authtime="0" dnstime="1" aptptime="269" cattime="279" avscantime="0" fullreqtime="99238" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36" exceptions="" category="177" reputation="trusted" categoryname="Content Server" content-type="application/octet-stream" reason="range"

secure:/root # grep spotify /var/log/http.log|grep 'statuscode="429"'|tail -1
2021:04:26-11:32:48 secure httpproxy[5914]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="POST" srcip="192.168.1.142" dstip="35.188.42.15" user="" group="" ad_domain="" statuscode="429" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="3" request="0xcfac0a00" url="https://sentry.io/api/114855/envelope/?sentry_key=de32132fc06e4b28965ecf25332c3a25&sentry_version=7" referer="">https://open.spotify.com/" error="" authtime="0" dnstime="1" aptptime="333" cattime="519" avscantime="5037" fullreqtime="56467" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36" exceptions="" category="105" reputation="trusted" categoryname="Business" sandbox="-" content-type="application/octet-stream"

secure:/root # grep spotify /var/log/http.log|grep 'statuscode="504"'|tail -1
2021:04:26-07:33:06 secure httpproxy[5914]: id="0002" severity="info" sys="SecureWeb" sub="http" name="web request blocked" action="block" method="GET" srcip="192.168.1.142" dstip="35.186.224.44" user="" group="" ad_domain="" statuscode="504" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="2851" request="0xa4d6300" url="https://guc-dealer.spotify.com/?access_token=BQBRtJmS-HZiACsBVUERl6gzfFn3OqBUL9pc-z4yjYdGYMzWJ731bKcaCq8XEAL_YZrEk4rEw5aLrQw6c-5Nk0yuZpNIAwkolLI141TDuZohiMm1oby672NY9LFAPNRxPGtKCHE9Q8w-Up3kDu6thaoNF2zrQrcp0RlUQnji0NLQhJd73yIkqzw0s2N7YFEEJviCeLTp_gY26xDllpoEkSVSS7-36kMayqZxRGy2uPt1jy-5pe2eCOEEHgO045ibsvwAUeR9UJI1Ewvqsfo1KiPo-Efr8ETJbOguBfvfDbQ" referer="" error="Timeout while reading response from Server" authtime="0" dnstime="1" aptptime="335" cattime="310" avscantime="0" fullreqtime="60741018" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36" exceptions="" category="129" reputation="trusted" categoryname="Media Downloads"



Better title
[edited by: BAlfson at 7:46 PM (GMT -7) on 27 Apr 2021]