Troubleshooting Web Sites with Web Browsers
Troubleshooting web sites with
web browsers
LIR HEAnet User Group for Libraries
DCU June 7
th
2016
glenn.wearen@heanet.ie
Agenda
•
Introduction and information about LIR
•
HTTP Primer / CSS & JS updates
•
Troubleshooting a website with Google
Chrome
•
Monitoring? routes
Introduction / about LIR
LIR HEAnet User Group for Libraries aims to explore and
develop awareness of electronic information resources and
to promote the use of the HEAnet network in exploiting
these resources. Established in
1993,
LIR:
•
Provides a forum for discussion on the use of electronic
resources and networks by HEI libraries
•
Assists in development and training for their effective use
•
Encourages communication and co-operation between
member libraries in accessing electronic resources.
Introduction / about LIR
•
Workshops
–
Digital Marketing May 2013
–
HEAnet Services Day
November 2014
–
Tech Tools for Teaching
December 2014
•
Annual Seminar
Introduction / about LIR
•
Why is there a LIR HEAnet User Group
–
Originally, the group was established to encourage
use of the internet and the HEAnet network by
libraries.
–
Today, the group continues this mission but has
extended to include HEAnet services, and advanced
internet tech.
Filesender
Edustorage
Edugate
Eduroam
Multimedia services (event recording/live streaming/video
or desktop conferencing/video hosting)
Introduction /about LIR
•
Why this workshop?
–
Equip libraries with the web skills to get the
most out of their web-resources
–
Library website
Hosted or on-campus services (proxy, Catalogue,
Discovery Service, Institutional repository)
–
E-resources
Ebooks, journals etc.
Introduction / about LIR
•
Who monitors their web-resources?
–
User access rates?
–
Availability (up/down)?
–
Error rates?
–
Response times?
–
Off-campus login experience?
•
Not all users complain, and when they do, problems
can take weeks to resolve.
–
This workshop will provide you with evidence to support
issues you raise with providers and help you think more
proactively about discovering issues before users do.
HTTP Primer
Fundamentals of a web-request
1.
User requests web-site with a browser.
request is delivered via HTTP Protocol
2.
Response is also delivered via HTTP
Protocol, including a HTML document
Document is rendered by the browser.
HTTP Primer
1.
User requests web-site with a browser
GET
/drupal/?q=aboutus HTTP/1.1
Host:
lirgroup.heanet.ie Connection: keep-alive
Accept
: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;
Upgrade-Insecure-Requests:
1
User-Agent:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/50.0.2661.102 Safari/537.36
DNT:
1
Referer:
http://lirgroup.heanet.ie/drupal/?q=seminars
Accept-Encoding:
gzip, deflate, sdch Accept-Language: en-US,en;q=0.8
Cookie:
__utma=124274707.2107404499.1464600275.1464600275.1464609843.2;
__utmc=124274707;_utmz=124274707.1464609843.2.2.utmcsr=google|utmccn=(organic)|utmcmd=
organic|utmctr=(not%20provided);
Request HEADER can be spoofed
Request HEADER can be altered by intermediary devices (proxy/web-cache)
•
Some headers are part of original HTTP specification, others are browser specific or
open to use by other browsers
HTTP Primer
1.
Response generated by web-server
HTTP/1.1 200 OK
Server
: nginx
Date
: Tue, 31 May 2016 10:46:02 GMT
Content-Type
: text/html; charset=utf-8
Transfer-Encoding:
chunked
Connection
: keep-alive
X-Drupal-Cache:
HIT
Etag
: "1464665626-1"
Content-Language:
en
Link:
</drupal/?q=node/2>; rel="shortlink",</drupal/?q=aboutus>; rel="canonical"
X-Generator:
Drupal 7 (http://drupal.org)
Cache-Control:
public, max-age=0
Expires:
Sun, 19 Nov 1978 05:00:00 GMT
Vary:
Cookie,Accept-Encoding
Content-Encoding:
gzip
Last-Modified:
Tue, 31 May 2016 03:33:46 GMT
X-Powered-By:
PleskLin
Response can be spoofed if request is intercepted by malicious user
Response can be altered by intermediary devices, e.g. proxy, web-cache
Some headers are part of HTTP standard, others are proprietary.
HTTP Primer
<!DOCTYPE
html
PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd"><html
xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" version="XHTML+RDFa 1.0" dir="ltr"
xmlns:skos="http://www.w3.org/2004/02/skos/core#" xmlns:xsd="http://www.w3.org/2001/XMLSchema#">
<
head
profile="http://www.w3.org/1999/xhtml/vocab">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link rel="shortcut icon" href="http://lirgroup.heanet.ie/drupal/misc/favicon.ico" type="image/vnd.microsoft.icon" />
<meta name="HandheldFriendly" content="true" />
<link rel="shortlink" href="/drupal/?q=node/2" />
<meta name="Generator" content="Drupal 7 (http://drupal.org)" />
<link rel="canonical" href="/drupal/?q=aboutus" />
<meta name="MobileOptimized" content="width" />
<title>About LIR | Drupal</title>
<style type="text/css" media="all">@import url("http://lirgroup.heanet.ie/drupal/modules/system/system.base.css?ngqedd><link type="text/css" rel="stylesheet" href="http://lirgroup.heanet.ie/drupal/…/ie.css?ngqedd" media="all”
<script type="text/javascript" src="http://lirgroup.heanet.ie/drupal/misc/jquery.js?v=1.4.4"></script>
<script type="text/javascript" src="http://lirgroup.heanet.ie/drupal/misc/drupal.js?ngqedd"></script
</
head
>
<
body
class="html front not-logged-in one-sidebar sidebar-second page-node page-node- page-node-2 node-type-page" >
<div id="skip-link”>
<a href="#main-content" class="element-invisible element-focusable">Skip to main content</a>
</div>
<div id="header”>
<div id="header-inside" class="container_12 clearfix”>
<div id="header-inside-left" class="grid_8”>
<a href="/drupal/" title="Home”><img src="http://lirgroup.heanet.ie/…/lir-logo.png" alt="Home”/></a>
</div>
<div id="header-inside-right" class="grid_4">
…
</div>
</div>
</body>
</
html
>
HTTP Primer
(Selected) HTTP Response Codes:
•
200 OK
•
301 Not Modified
•
302 Found (elsewhere)
•
401 Not Authorized
•
403 Forbidden
•
404 Not Found
•
502 Bad Gateway
•
503 Internal Server Error
•
Or no response at all!
HTTP Primer
HTTP GET/POST/HEAD
•
GET used primarily for browsing pages, it is the
default for links.
•
POST used for submitting forms or content
–
submitted data present in the body of the request
–
GET can also be used with forms with query
parameter
e.g. http://…/myform?username=joebloggs
•
HEAD is used to fetch header content only
–
e.g checking cache validity
HTTP Primer
Cookies
•
When a web-server returns a response with “set-cookie” the
browser is expected to save a cookie and send it with every
subsequent request until;
–
It expires
–
Web server resets it with new value or new expiry date
•
Cookie content is usually encrypted (not SSL encryption) so
that content is only understood by web-server.
–
Cookies can be guessed, but would require multiple requests
•
Cookies parameter set by web-server
–
expiry (date in the future when cookie expires)
–
http-only (only to be used by the browser, not to be used by
JavaScript etc.)
–
domain
e.g. ucd.ie (no domain means the current web-server –e.g.
lirgroup.heanet.ie
–
Browser will only send cookies back to hosts within that domain
HTTP Primer
Cookies
•
Third party cookies are those included in a
response for servers in a different domain than
the current domain
–
A web-server may include additional 3
rd
party cookie
in its response.
–
Content for parts of a web-page (e.g. image) may be
hosted on another server, browser requests for that
content might include a set-cookie response which
are considered 3
rd
party by the browser
•
20-25% of users have disabled 3
rd
party cookies
in their browser.
HTTP Primer
SSL
•
A secure webserver has a private encryption key and a public certificate,
known as a keypair. The public certificate is signed by a Certificate
Authority (CA). CA’s are preconfigured in all browsers.
•
When requesting a https resource…
1.
the server sends its public key certificate in the response (and
optionally intermediate CA certificates).
2.
The browser checks that this certificate
is signed by one of its pre-configured CA’s,
is still valid (not revoked),
matches the website being accessed.
3.
A session encryption key is generated for subsequent requests.
•
Certificates have a Subject Name and Subject Alternative
Name, the alternative name is used for aliases of the website
–
This is how proxy servers can act act as a middleman between the browser
and server
CSS & JS
Dynamic documents
•
CSS and JavaScript used to dynamically render a
web-page (e.g. responsive-design) by
manipulating the Document Object Model (DOM).
•
XMLHttpRequest used to send requests to the
webserver from within a document (e.g. type-
ahead fields)
–
Saves mulitple rounds trips by the user to the
webserver
CSS & JS
•
A failure to deliver a CSS file or JS file to a
browser will result in usability issues.
•
Failures caused by
–
human error (e.g. 403)
–
Browser cache not updated (browser or server)
–
Timeout delivering content
–
Server error (e.g. 503)
–
Proxy error, or untested site via proxy
CSS & JS
•
A failure to render a page can be caused by
CSS or JS errors
•
Failures caused by
–
human error (e.g. typographical)
–
Old browsers not supporting modern feature
–
Untested modern browser
–
Too much content / low powered client
Google Chrome Developer Tools
•
Inspect Request, Response
–
Headers
–
Body
–
Cookies
•
Debug JavaScript Errors
•
Save as HAR file, replay later
–
http://www.softwareishard.com/har/viewer/
–
Minus **passwords**
Monitoring
•
Open-source
–
Icinga/Nagios
Use with Graphite to record historical performance,
or use Cactii/Munin standalone
•
Commercial
–
Pingdom
Free account allows monitoring of one service
–
New Relic
Monitoring
•
What to monitor (in order of complexity)
1.
Basic up/down
2.
Responsiveness/latency
3.
Errors loading or rendering pages
4.
Simulated login to protected resources
5.
Search result performance.
Explore troubleshooting web sites with web browsers in the context of LIR HEAnet User Group for Libraries. This workshop covers topics such as CSS & JS updates, monitoring routes, user access rates, and more to equip libraries with essential web skills for optimizing electronic resources.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Troubleshooting web sites with web browsers LIR HEAnet User Group for Libraries DCU June 7th2016 glenn.wearen@heanet.ie
Agenda Introduction and information about LIR HTTP Primer / CSS & JS updates Troubleshooting a website with Google Chrome Monitoring? routes
Introduction / about LIR LIR HEAnet User Group for Libraries aims to explore and develop awareness of electronic information resources and to promote the use of the HEAnet network in exploiting these resources. Established in 1993, LIR: Provides a forum for discussion on the use of electronic resources and networks by HEI libraries Assists in development and training for their effective use Encourages communication and co-operation between member libraries in accessing electronic resources.
Introduction / about LIR Workshops Digital Marketing May 2013 HEAnet Services Day November 2014 Tech Tools for Teaching December 2014 Annual Seminar
Introduction / about LIR Why is there a LIR HEAnet User Group Originally, the group was established to encourage use of the internet and the HEAnet network by libraries. Today, the group continues this mission but has extended to include HEAnet services, and advanced internet tech. Filesender Edustorage Edugate Eduroam Multimedia services (event recording/live streaming/video or desktop conferencing/video hosting)
Introduction /about LIR Why this workshop? Equip libraries with the web skills to get the most out of their web-resources Library website Hosted or on-campus services (proxy, Catalogue, Discovery Service, Institutional repository) E-resources Ebooks, journals etc.
Introduction / about LIR Who monitors their web-resources? User access rates? Availability (up/down)? Error rates? Response times? Off-campus login experience? Not all users complain, and when they do, problems can take weeks to resolve. This workshop will provide you with evidence to support issues you raise with providers and help you think more proactively about discovering issues before users do.
HTTP Primer Fundamentals of a web-request 1. User requests web-site with a browser. request is delivered via HTTP Protocol 2. Response is also delivered via HTTP Protocol, including a HTML document Document is rendered by the browser.
HTTP Primer 1. User requests web-site with a browser GET /drupal/?q=aboutus HTTP/1.1 Host: lirgroup.heanet.ie Connection: keep-alive Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*; Upgrade-Insecure-Requests: 1 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36 DNT: 1 Referer: http://lirgroup.heanet.ie/drupal/?q=seminars Accept-Encoding: gzip, deflate, sdch Accept-Language: en-US,en;q=0.8 Cookie: __utma=124274707.2107404499.1464600275.1464600275.1464609843.2; __utmc=124274707;_utmz=124274707.1464609843.2.2.utmcsr=google|utmccn=(organic)|utmcmd= organic|utmctr=(not%20provided); Request HEADER can be spoofed Request HEADER can be altered by intermediary devices (proxy/web-cache) Some headers are part of original HTTP specification, others are browser specific or open to use by other browsers
HTTP Primer 1. Response generated by web-server HTTP/1.1 200 OK Server: nginx Date: Tue, 31 May 2016 10:46:02 GMT Content-Type: text/html; charset=utf-8 Transfer-Encoding: chunked Connection: keep-alive X-Drupal-Cache: HIT Etag: "1464665626-1" Content-Language: en Link: </drupal/?q=node/2>; rel="shortlink",</drupal/?q=aboutus>; rel="canonical" X-Generator: Drupal 7 (http://drupal.org) Cache-Control: public, max-age=0 Expires: Sun, 19 Nov 1978 05:00:00 GMT Vary: Cookie,Accept-Encoding Content-Encoding: gzip Last-Modified: Tue, 31 May 2016 03:33:46 GMT X-Powered-By: PleskLin Response can be spoofed if request is intercepted by malicious user Response can be altered by intermediary devices, e.g. proxy, web-cache Some headers are part of HTTP standard, others are proprietary.
HTTP Primer <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" version="XHTML+RDFa 1.0" dir="ltr" xmlns:skos="http://www.w3.org/2004/02/skos/core#" xmlns:xsd="http://www.w3.org/2001/XMLSchema#"> <head profile="http://www.w3.org/1999/xhtml/vocab"> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta name="viewport" content="width=device-width, initial-scale=1" /> <link rel="shortcut icon" href="http://lirgroup.heanet.ie/drupal/misc/favicon.ico" type="image/vnd.microsoft.icon" /> <meta name="HandheldFriendly" content="true" /> <link rel="shortlink" href="/drupal/?q=node/2" /> <meta name="Generator" content="Drupal 7 (http://drupal.org)" /> <link rel="canonical" href="/drupal/?q=aboutus" /> <meta name="MobileOptimized" content="width" /> <title>About LIR | Drupal</title> <style type="text/css" media="all">@import url("http://lirgroup.heanet.ie/drupal/modules/system/system.base.css?ngqedd> <link type="text/css" rel="stylesheet" href="http://lirgroup.heanet.ie/drupal/ /ie.css?ngqedd" media="all <script type="text/javascript" src="http://lirgroup.heanet.ie/drupal/misc/jquery.js?v=1.4.4"></script> <script type="text/javascript" src="http://lirgroup.heanet.ie/drupal/misc/drupal.js?ngqedd"></script </head> <body class="html front not-logged-in one-sidebar sidebar-second page-node page-node- page-node-2 node-type-page" > <div id="skip-link > <a href="#main-content" class="element-invisible element-focusable">Skip to main content</a> </div> <div id="header > <div id="header-inside" class="container_12 clearfix > <div id="header-inside-left" class="grid_8 > <a href="/drupal/" title="Home ><img src="http://lirgroup.heanet.ie/ /lir-logo.png" alt="Home /></a> </div> <div id="header-inside-right" class="grid_4"> </div> </div> </body> </html>
HTTP Primer (Selected) HTTP Response Codes: 200 OK 301 Not Modified 302 Found (elsewhere) 401 Not Authorized 403 Forbidden 404 Not Found 502 Bad Gateway 503 Internal Server Error Or no response at all!
HTTP Primer HTTP GET/POST/HEAD GET used primarily for browsing pages, it is the default for links. POST used for submitting forms or content submitted data present in the body of the request GET can also be used with forms with query parameter e.g. http:// /myform?username=joebloggs HEAD is used to fetch header content only e.g checking cache validity
HTTP Primer Cookies When a web-server returns a response with set-cookie the browser is expected to save a cookie and send it with every subsequent request until; It expires Web server resets it with new value or new expiry date Cookie content is usually encrypted (not SSL encryption) so that content is only understood by web-server. Cookies can be guessed, but would require multiple requests Cookies parameter set by web-server expiry (date in the future when cookie expires) http-only (only to be used by the browser, not to be used by JavaScript etc.) domain e.g. ucd.ie (no domain means the current web-server e.g. lirgroup.heanet.ie Browser will only send cookies back to hosts within that domain
HTTP Primer Cookies Third party cookies are those included in a response for servers in a different domain than the current domain A web-server may include additional 3rdparty cookie in its response. Content for parts of a web-page (e.g. image) may be hosted on another server, browser requests for that content might include a set-cookie response which are considered 3rdparty by the browser 20-25% of users have disabled 3rdparty cookies in their browser.
HTTP Primer SSL A secure webserver has a private encryption key and a public certificate, known as a keypair. The public certificate is signed by a Certificate Authority (CA). CA s are preconfigured in all browsers. When requesting a https resource 1. the server sends its public key certificate in the response (and optionally intermediate CA certificates). 2. The browser checks that this certificate is signed by one of its pre-configured CA s, is still valid (not revoked), matches the website being accessed. 3. A session encryption key is generated for subsequent requests. Certificates have a Subject Name and Subject Alternative Name, the alternative name is used for aliases of the website This is how proxy servers can act act as a middleman between the browser and server
CSS & JS Dynamic documents CSS and JavaScript used to dynamically render a web-page (e.g. responsive-design) by manipulating the Document Object Model (DOM). XMLHttpRequest used to send requests to the webserver from within a document (e.g. type- ahead fields) Saves mulitple rounds trips by the user to the webserver
CSS & JS A failure to deliver a CSS file or JS file to a browser will result in usability issues. Failures caused by human error (e.g. 403) Browser cache not updated (browser or server) Timeout delivering content Server error (e.g. 503) Proxy error, or untested site via proxy
CSS & JS A failure to render a page can be caused by CSS or JS errors Failures caused by human error (e.g. typographical) Old browsers not supporting modern feature Untested modern browser Too much content / low powered client
Google Chrome Developer Tools Inspect Request, Response Headers Body Cookies Debug JavaScript Errors Save as HAR file, replay later http://www.softwareishard.com/har/viewer/ Minus **passwords**
Monitoring Open-source Icinga/Nagios Use with Graphite to record historical performance, or use Cactii/Munin standalone Commercial Pingdom Free account allows monitoring of one service New Relic
Monitoring What to monitor (in order of complexity) 1. Basic up/down 2. Responsiveness/latency 3. Errors loading or rendering pages 4. Simulated login to protected resources 5. Search result performance.
