NHS Infrastructure (projects)

open forum
21

Amusing data antics anecdote…

Wanted a squizz at the ODS data… downloaded it from TRUD.

It’s a zip archive with all the documentation and data in it as CSV files. Despite being a bit pointless inside another archive, the data files are all individual archives.

They’re titled with a suffix of .zip

awilkins@motoko:~/data/ODS_weekly/Data$ unzip egpcur.zip 
Archive:  egpcur.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of egpcur.zip or
        egpcur.zip.zip, and cannot find egpcur.zip.ZIP, period.

They’re actually RAR archives. 7-zip File Manager correctly recognizes this and opens the archive up. I wonder how many automated ETL processes this has stymied…

22

that doesn’t make sense …
if they are not practicing how can they be assigned to a practice?
(everyone need to stick to their jobs)

23

It doesn’t make sense.
Would you use it to find the practice a patient belongs to (if the patient was registered to a retired GP). Still seems wrong though.

Adrian… I think that maybe where my warnings are coming from.

24

Download a zip, parse contents, mangle for diff and fire changes into a DB?
I’d reach for Pentaho Kettle. When you’ve built a job it can be exported
and run to a schedule.

Rob

25

Is that open source? It’s not clear from the website

Wondering if it’s better to process the files using something like this and output a diff?

Currently I have:

  1. http get
  2. unzip file
  3. for each line in file check to see (via REST read) entity exists or has been updated
  4. if not exists then create (REST post) else update (REST put)

The rest calls @ 3 are a bit expensive - probably better to keep a copy of the last file processed and use that for the diffs.

May use the data in the file as triggers only and use LDAP to get the actual data (seems to be better quality) - so a diff would work still.

26

http://community.pentaho.com/projects/data-integration/

27

as promised here is a link:

myModel1

i have decided to drip feed myModel as it is really, really complicated (it's not possible to overload it

with data but i suspect it could be possible to overload the people trying to understand it)

notes:

1.all health transactions should be assigned to a PATIENT pathway (even if it is just the NHS TimeLine,

uniquely ID'd bi the NHS Number)

2. care pathways are color coded e.g. a child PATIENT pathway where the care pathways is autism might

be coloured PURPLE (all the BI technologies make heavy use of color including Tableau ;-))

IF THIS IS OF INTEREST TO YOU PLEASE LET ME KNOW AS I INTEND TO PASSWORD PROTECT

(i am 'appy to be open with those that i trust, but i am not an idiot, nothing is free and in myWorld the

most important thing "i can do" is to develop my data model and play with it using Qlik tech)

i use Qlik technology to model data for and develop PATIENT health apps, I am comfortable with it,

Qlik Sense app tech is in it's early stage of development, they really are a way

of rendering their dashboards on mobile devices, they are not that "appy" yet but THEY are

able to invest in and "DO" the right things

Qlik will not share their plans4the future, why should they? they live in the real world, the competitive

world, they must stay ahead of the competition :-) but i trust they are "heading" in the right direction ...

(i.e. they do not have the same usability as, say, a bank app giving customers access to their

financial transactions) - i am looking @ other technologies

ps thanx 2Google for allowing me to share my model with others in a similar world to me

NEXT ? integarted Care Teams, then integarted Care Models

On 17 March 2016 at 20:59 Rob Dyke <discourse-system@openhealthhub.org> wrote:

robdykedotcom

March 17

http://community.pentaho.com/projects/data-integration/

Visit Topic or reply to this email to respond

In Reply To

mayfield.g.kev

March 17Is that open source? It’s not clear from the website Wondering if it’s better to process the files using something like this and output a diff? Currently I have: http get unzip file for each line in file check to see (via REST read) entity exists or has been updated if not exists then crea…

Visit Topic or reply to this email to respond

To stop receiving notifications for this particular topic, click here. To unsubscribe from these emails, change your user preferences

28

Quick update.

Have found a better source for the data. NHS Monthly amendments which means less processing to work out what has changed http://systems.hscic.gov.uk/data/ods/datadownloads/monthamend

Also have loaded several files into the HAPI FHIR Database. Quite impressed with the results (well it’s great for a web developer but may not so for SQL devs.) A search for GP’s in Erewash CCG is reduced to

GET http://localhost:8080/hapi-fhir-jpaserver/baseDstu2/Practitioner?organization.partof.identifier=urn:fhir.nhs.uk/id/ODSOrganisationCode|03X

Which returns JSON representation of GP’s in the CCG. No messing with SQL or building the API layer.

30

Kev, a patient isn’t registered to a GP at all but to a practice. This caused a lot of problems over the years with legacy systems expecting an individual not an org, but recently they seem to have reintroduced a relationship to a named “lead” GP (who is probably the senior partner in the practice) and may well not be the person that the patient regards as “their” doctor.

31

You’re correct (did have to check a spine response to confirm).

I’d not noticed before, must have got used to seeing Patients ManaginOrganization and Practitioner everywhere

32

and in non FHIR terms … Patients Practice and GP.

34

Just as a side note, a colleague has recently setup a cloud forum specifically relating to the NHS. We are working on digitising NHS projects in conjunction with GDS and NHS Digital and will be sharing our experiences https://www.nhscloudcommunity.co.uk/

This all began at the recent AWS Summit in London where it was recognised that there was a need for all NHS authorities, Department of Health, Public Health England and their partners to work with each other and share ideas and learning to get the best outcomes of using cloud technologies within our sector.

35

Have gained some very useful cloudy experience here at DWP recently.

36

same thing here… i’ve been lurking here for a while (for a while i’m just sitting home doing nothing but searching for rx coupons because of my health issues and for lack of money, so i decided to learn a little bit while doing nothing literally) but now i decided to register since i wanted to talk with other people and i also had some questions. but definitely already gained lots of experience and useful info here!

1 Like
37

@Lonsward thanks - glad you like the forum… please do invite other people to the community that you think would benefit from the info we share here…

Marcus