Monthly Publishing System (MPS) Workshop Overview
Monthly Publishing System (MPS)
Developer Workshop
25 August, 2015
High-level Overview
•
Introduction
•
Preliminary discussion session
•
Overview of the MPS
–
Lunch
•
Presentation and Workshop
•
Wrap-up discussion session
–
Finish
(‘Break’ for those that want to continue)
•
Informal Workshop
2
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
Preliminary Discussion Session
•
Results of the Developer Survey
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
3 of 3
MPS Overview
•
Design Objectives
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
4 of 4
Monthly Publishing System (MPS)
Developer Workshop
25 August, 2015
Workshop Agenda
•
Deliverables
•
Quality
–
Validation (testing)
–
Verification (counting)
•
Tracking Changes
•
Options for formatting
•
Options for querying
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
6 of 6
Workshop Format
•
Keep it interactive!
–
The learning should go both ways.
–
Some questions may need to be deferred till the
informal session.
•
The objective is to be ‘hands-on’ but the format
may require some flexibility.
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
7
MPS Overview
•
What does it deliver?
–
Reports
–
Publications
•
How does it deliver?
–
MPS API
–
Summary of Changes
–
SQLite
–
JSON
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
8
What does the MPS deliver?
•
Publications
–
Summary of Changes
–
Hardcopy schedules (PDF)
–
Legislative Instruments
–
Website / offline website / mobile website
–
SQLite / SQL script
–
Text files
–
API
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
9
Key Challenges
•
Grouping
–
Hardcopy vs legislative instruments vs …?
•
Sorting
–
The primary specification is the 30 year old hardcopy.
•
Ordering
–
Varies – a recent example is notes to restrictions.
•
Identifying
–
The lack of permanent IDs is not optimal.
The fundamental issue is coping with the complexity of
the data in the context of each deliverable!
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
10
Other challenges
•
The schema is big and there are a lot of places
for data to hide.
•
No one asks what will work best for developers
before they amend the legislation.
•
The first day of the month is non-negotiable.
•
IE 6…
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
11
The solution
•
Both ends are volatile.
•
Neither end is under our control.
•
The business logic is expensive and fragile
therefore duplication must be avoided.
•
Stability is needed somewhere, so the only
option for the MPS is to create an API in
between the source data and the deliverables.
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
12
MPS processing
•
Start with the PBS XML and extensive validation.
–
Well-formedness
–
Schema
–
Schematron
–
Internal format validation
•
Validation isn’t enough.
–
No errors doesn’t guarantee no problems.
–
Validation errors aren’t always easy to interpret.
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
13
MPS Verification
•
Count everything (between months and each
version)
–
Generic Drug
–
PBS Item
–
PBS Product
–
Schedule
•
Reports:
https://dev.pbs.gov.au/docs/reference/pbsxml_reports.html
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
14
MPS Statistics Report
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
15
MPS Summary of Changes
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
16
MPS Summary of Changes
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
17
MPS Summary of Changes
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
18
MPS Summary of Changes
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
19
MPS Summary of Changes
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
20
Processing issues
•
Data overlap
–
Much of the source data looks the same.
•
It isn’t easy to determine which values are correct or
when you have used the wrong value.
–
Correct decisions may become wrong.
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
21
Protection from volatility
•
MPS API
–
Consequences of change are always limited to the
distance between the source of the change and the
API.
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
22
Example: Summary of Changes
•
Addition Items
–
Pull data by item code
•
Use service: /0.3/items/{code}•
Addition Brands
–
Pull date by brand code
•
Use service: /0.3/brands/{code}•
Addition Notes
–
Pull data by note code
–
Use service: /0.3/notes/{code}MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
23
Example: Summary of Changes
•
Deletion Items
–
Pull last month of data by item code
•
Use service: /0.3/items/{code}?effectivedate=2015-07-01•
Deletion Brands
–
Pull last month date by brand code
•
Use service: /0.3/brands/{code}?effectivedate=2015-07-01•
Deletion Notes
–
Pull last month data by note code
•
Use service: /0.3/notes/{code}?effectivedate=2015-07-01MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
24
Example: Summary of Changes
•
Alteration
–
Brand name
–
Form strength
–
Prescriber group
–
Manufacturer
–
Restriction
–
Authority type
–
Max quantity
–
Pack quantity
–
Number of repeat
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
25
Example: Summary of Changes
•
Alteration (continue)
–
Pull data by item code
•
Current month:
/0.3/items/{code}?effectivedate=2015-08-01•
Last month:
/0.3/items/{code}?effectivedate=2015-07-01MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
26
Example: Summary of Changes
•
Advance notice
- Pull data by item code
•
Current month:
/0.3/items/{code}?effectivedate=2015-08-01- Check the element
•
AdvanceNotice.Type
•
AdvanceNotice.EffectiveDate
http://api.pbs.gov.au/0.3/items/8023G.xml?effectivedate=2015-08-
01&advancenotice=true
<AdvanceNotice.Type>
deletion
</AdvanceNotice.Type>
<AdvanceNotice.EffectiveDate>
2015-10-01
</AdvanceNotice.EffectiveDate>
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
27
MPS SQLite
•
Business Objectives
–
Create a portable, lightweight representation of the
data as an alternative to the text files.
•
Minimal dependencies.
•
Zero licensing issues.
•
The answer
–
SQLite database.
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
28
MPS SQLite
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
29
MPS JSON
•
A lightweight approach suited to modern web
architectures.
–
JSON connects directly to the API.
–
Use Javascript to process in the browser.
–
Minimal learning required.
–
Very little infrastructure required.
–
More usable than the text files but based on modern
technology.
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
30
MPS JSON
JSON Syntax
•
Constructed from the XML syntax
–
Namespace is ignored
•
Using the following rules:
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
31
MPS JSON
•
XML
•
JSON
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
32
MPS API
•
How mature is the API?
–
Used by the MPS since July 2011 opened to the public in
August 2012.
•
Powers the general PBS Website.
•
Powers the mobile PBS Website.
–
Recently upgraded to 0.3 (4
th
version).
•
Now used for the hardcopy and Summary of Changes.
–
Undergoes extensive automated testing each month.
–
Is manually reviewed by a lot of people each month
(community and industry).
–
Previous API versions are kept up-to-date until all users
have migrated forward.
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
33
MPS API
Where is it?
http://api.pbs.gov.au
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
34
MPS API
How to access it?
•
Sign up by sending an email to:
pbs.websmaster@health.gov.au
•
You will be given a unique API Key
–
To give you access to the API
–
To protect the system from abuse
–
To allow us to optimise the data based on real usage
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
35
MPS API
What formats are supported?
•
The API can deliver the PBS data
–
a
s
X
M
L
–
O
r
a
s
a
J
S
O
N
o
b
j
e
c
t
•
Also supports JSONP
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
36
More information
•
API Documentation
https://dev.pbs.gov.au/docs/
•
To familiarise yourself with API
http://api.pbs.gov.au/howto.html
•
Developer online forum
https://dev.pbs.gov.au/discussions.html
MPS Workshop – August 25, 2016
h
t
t
p
s
:
/
/
d
e
v
.
p
b
s
.
g
o
v
.
a
u
/
d
o
c
s
/
37
High-level overview, preliminary discussion, design objectives, and workshop agenda of the Monthly Publishing System Developer Workshop held on August 25, 2016. Delve into the format, deliverables, challenges, and key features of MPS, including reports, publications, API, and more.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Monthly Publishing System (MPS) Developer Workshop 25 August, 2015
High-level Overview Introduction Preliminary discussion session Overview of the MPS Lunch Presentation and Workshop Wrap-up discussion session Finish ( Break for those that want to continue) Informal Workshop 2 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
Preliminary Discussion Session Results of the Developer Survey 3 of 3 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS Overview Design Objectives 4 of 4 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
Monthly Publishing System (MPS) Developer Workshop 25 August, 2015
Workshop Agenda Deliverables Quality Validation (testing) Verification (counting) Tracking Changes Options for formatting Options for querying 6 of 6 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
Workshop Format Keep it interactive! The learning should go both ways. Some questions may need to be deferred till the informal session. The objective is to be hands-on but the format may require some flexibility. 7 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS Overview What does it deliver? Reports Publications How does it deliver? MPS API Summary of Changes SQLite JSON 8 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
What does the MPS deliver? Publications Summary of Changes Hardcopy schedules (PDF) Legislative Instruments Website / offline website / mobile website SQLite / SQL script Text files API 9 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
Key Challenges Grouping Hardcopy vs legislative instruments vs ? Sorting The primary specification is the 30 year old hardcopy. Ordering Varies a recent example is notes to restrictions. Identifying The lack of permanent IDs is not optimal. The fundamental issue is coping with the complexity of the data in the context of each deliverable! 10 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
Other challenges The schema is big and there are a lot of places for data to hide. No one asks what will work best for developers before they amend the legislation. The first day of the month is non-negotiable. IE 6 11 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
The solution Both ends are volatile. Neither end is under our control. The business logic is expensive and fragile therefore duplication must be avoided. Stability is needed somewhere, so the only option for the MPS is to create an API in between the source data and the deliverables. 12 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS processing Start with the PBS XML and extensive validation. Well-formedness Schema Schematron Internal format validation Validation isn t enough. No errors doesn t guarantee no problems. Validation errors aren t always easy to interpret. 13 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS Verification Count everything (between months and each version) Generic Drug PBS Item PBS Product Schedule Reports: https://dev.pbs.gov.au/docs/reference/pbsxml_reports.html https://dev.pbs.gov.au/docs/reference/pbsxml_reports.html 14 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS Statistics Report 15 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS Summary of Changes 16 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS Summary of Changes 17 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS Summary of Changes 18 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS Summary of Changes 19 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS Summary of Changes 20 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
Processing issues Data overlap Much of the source data looks the same. It isn t easy to determine which values are correct or when you have used the wrong value. Correct decisions may become wrong. 21 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
Protection from volatility MPS API Consequences of change are always limited to the distance between the source of the change and the API. 22 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
Example: Summary of Changes Addition Items Pull data by item code Use service: /0.3/items/{code} Addition Brands Pull date by brand code Use service: /0.3/brands/{code} Addition Notes Pull data by note code Use service: /0.3/notes/{code} 23 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
Example: Summary of Changes Deletion Items Pull last month of data by item code Use service: /0.3/items/{code}?effectivedate=2015-07-01 Deletion Brands Pull last month date by brand code Use service: /0.3/brands/{code}?effectivedate=2015-07-01 Deletion Notes Pull last month data by note code Use service: /0.3/notes/{code}?effectivedate=2015-07-01 24 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
Example: Summary of Changes Alteration Brand name Form strength Prescriber group Manufacturer Restriction Authority type Max quantity Pack quantity Number of repeat 25 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
Example: Summary of Changes Alteration (continue) Pull data by item code Current month: /0.3/items/{code}?effectivedate=2015-08-01 Last month: /0.3/items/{code}?effectivedate=2015-07-01 26 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
Example: Summary of Changes Advance notice - Pull data by item code Current month: - Check the element AdvanceNotice.Type AdvanceNotice.EffectiveDate /0.3/items/{code}?effectivedate=2015-08-01 http://api.pbs.gov.au/0.3/items/8023G.xml?effectivedate=2015-08- 01&advancenotice=true <AdvanceNotice.Type>deletion </AdvanceNotice.Type> <AdvanceNotice.EffectiveDate>2015-10-01 </AdvanceNotice.EffectiveDate> 27 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS SQLite Business Objectives Create a portable, lightweight representation of the data as an alternative to the text files. Minimal dependencies. Zero licensing issues. The answer SQLite database. 28 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS SQLite 29 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS JSON A lightweight approach suited to modern web architectures. JSON connects directly to the API. Use Javascript to process in the browser. Minimal learning required. Very little infrastructure required. More usable than the text files but based on modern technology. 30 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS JSON JSON Syntax Constructed from the XML syntax Namespace is ignored Using the following rules: 31 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS JSON XML JSON 32 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS API How mature is the API? Used by the MPS since July 2011 opened to the public in August 2012. Powers the general PBS Website. Powers the mobile PBS Website. Recently upgraded to 0.3 (4th version). Now used for the hardcopy and Summary of Changes. Undergoes extensive automated testing each month. Is manually reviewed by a lot of people each month (community and industry). Previous API versions are kept up-to-date until all users have migrated forward. 33 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS API Where is it? http://api.pbs.gov.au 34 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS API How to access it? Sign up by sending an email to: pbs.websmaster@health.gov.au You will be given a unique API Key To give you access to the API To protect the system from abuse To allow us to optimise the data based on real usage 35 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
MPS API What formats are supported? The API can deliver the PBS data as XML Or as a JSON object Also supports JSONP 36 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
More information API Documentation https://dev.pbs.gov.au/docs/ To familiarise yourself with API http://api.pbs.gov.au/howto.html Developer online forum https://dev.pbs.gov.au/discussions.html 37 MPS Workshop August 25, 2016 https://dev.pbs.gov.au/docs/
