Fork me on GitHub
#clojure-uk
<
2020-02-16
>
Ben Hammond09:02:26

question for you data wonks; I want to be able to query a lat/lon Point for membership of a UK postcode using Postgis and st_covered_by(). I am therefore trying to get hold of org.locationtech.jts.geom.Polygon data for UK postcodes; I've got the data from https://www.ordnancesurvey.co.uk/business-government/products/code-point-open but this seems to only contain a single point for each postcode. I've also downloaded some data from https://www.opendoorlogistics.com/wp-content/uploads/Data/UK-postcode-boundaries-Jan-2015.zip which contains a bunch of .shp, .shx files, which I've not yet figured out what to do with. Am I barking up the wrong tree? How would you normally go about doing this?

Ben Hammond09:02:52

so, how can I process .shrp files using Clojure? presumably https://github.com/rmarianski/geoscript-clj is the sort of thing

benedek09:02:04

@rlj may be able to help with this i am guessing

rlj12:02:33

I don't know about that 🙂 I've dealt with getting geodata on a page with JS, rather than CLJ/S. The steps I've taken have been generally as follows: 1. Get a shapefile from somewhere. Postcode-level polygons are generally only available through a commercial licence. For our latest little project https://www.uswitch.com/gas-electricity/guides/how-green-is-your-region/ postal areas were sufficient. We got a shapefile from here: https://www.geolytix.co.uk/#!geodata 2. Get hold of tools for converting shapefiles to GeoJSON and manipulating GeoJSON/TopoJSON (GDAL and His Excellency Mike Bostock's topojson npm package respectively) 3. Turn your shapefile into GeoJSON (you'll need .shp and .shx files for this): ogr2ogr -f GeoJSON -t_srs EPSG:4326 PostalArea.geo.json PostalArea.shp 4. Convert your GeoJSON into much nicer TopoJSON, and optimize to taste. This is what we did: geo2topo postal_areas=PostalArea.geo.json | toposimplify -S 0.01 -f | topoquantize 1e4 | jq '.objects.postal_areas.geometries[] |= (. += {id: .properties.PostArea})' > PostalArea.topo.json 5. And then use your lovely new TopoJSON file in the front-end application of your choice (probably involving D3) I don't know how useful this is, but there you go.

benedek13:02:48

did not know about the how green is ur region. thx!

rlj16:02:55

Waldemar and I are part-time widget creators these days. See also https://www.uswitch.com/gas-electricity/guides/mind-the-price-cap-gap/

rlj16:02:54

We should probably write them in something other than ES6 but that's what we're used to now (on the front end anyway).

Ben Hammond10:02:24

> What do you call an empty parrot cage?

8
Ben Hammond10:02:17

> Anything you like, it won't repeat you?

tumbleweed 4
🧥 4
4
Ben Hammond10:02:13

ohh theres a zip code polygon zipfile for 'Merka at https://geodata.lib.berkeley.edu/catalog/TG10USZCTA5 I note 615.15 Mb

rlj12:02:33

I don't know about that 🙂 I've dealt with getting geodata on a page with JS, rather than CLJ/S. The steps I've taken have been generally as follows: 1. Get a shapefile from somewhere. Postcode-level polygons are generally only available through a commercial licence. For our latest little project https://www.uswitch.com/gas-electricity/guides/how-green-is-your-region/ postal areas were sufficient. We got a shapefile from here: https://www.geolytix.co.uk/#!geodata 2. Get hold of tools for converting shapefiles to GeoJSON and manipulating GeoJSON/TopoJSON (GDAL and His Excellency Mike Bostock's topojson npm package respectively) 3. Turn your shapefile into GeoJSON (you'll need .shp and .shx files for this): ogr2ogr -f GeoJSON -t_srs EPSG:4326 PostalArea.geo.json PostalArea.shp 4. Convert your GeoJSON into much nicer TopoJSON, and optimize to taste. This is what we did: geo2topo postal_areas=PostalArea.geo.json | toposimplify -S 0.01 -f | topoquantize 1e4 | jq '.objects.postal_areas.geometries[] |= (. += {id: .properties.PostArea})' > PostalArea.topo.json 5. And then use your lovely new TopoJSON file in the front-end application of your choice (probably involving D3) I don't know how useful this is, but there you go.

rlj12:02:46

And mornin!

👍 12
👋 8
Ben Hammond14:02:52

That's extremely helpful Thank you

rlj16:02:46

No probs.

yogidevbear23:02:04

Morning o/ Some "light" viewing for when you have a spare 2 hours. Well worth watching until the very end if you can https://www.youtube.com/watch?v=Lg3zq7Lrxl0

👍 8
picard-facepalm 4
Aleksander07:02:01

🙈 I would very much prefer not to. He's well known for transphobia, homophobia, etc, etc. plus being a geteway drug to alt-right communities/thinkers And if that didn't bother me much I would still be weary of accepting any advice from a "psychologist" who a) first got addicted to drugs b) then decided it's a good idea to seek alternative treatments for the addiction in Russia c) which almost got him killed in Russia. At the very least I would prefer his videos not being posted here ....

Gulli07:02:38

@U0MFQNXFZ Thanks, since I'm taking care of the wee one today (she's sick and so no nursery), I'll put it on in the background.

Conor08:02:54

Ah, the lobster guy. I would trust his conclusions about as far as I can throw them

yogidevbear08:02:00

Interesting responses. Apologies if the post offends you @UA2R84M28. Wasn't my intention at all ❤️ I don't know much about him, but I thought this particular lecture had some well presented thoughts towards the very end which I thought were actually a quite positive (i.e. not hateful, etc).

Gulli09:02:19

@U0MFQNXFZ We'll, many people do like and enjoy his content. Hopefully people are making up their own mind rather than listening to what others are telling them :)

otfrom09:02:08

I'm not sure I need to reward someone like him with clicks which earn him money. This is all very disappointing.

yogidevbear09:02:32

I've gone and deleted the post. Really wasn't intending to upset people like this

Wes Hall10:02:44

It's funny, I missed the initial post before it was deleted but the thread gives me enough to know exactly who was in the initial post.

Wes Hall11:02:51

I do think anybody getting too "into" this particular social commentator should do themselves a favour and listen to the podcast he did with Sam Harris. It's frustrating because they spend quite a lot of it discussing the definition of "truth", with (imo), Sam having a perfectly reasonable one, and 'other guy' having a batshit insane one. I like him on some subjects and I think his "rise to fame", was less about transphobia and more about government enforced rules of speech, but "dash of salt" does apply I think.

Gulli12:02:55

I think in general people should look into an assortion of videos if they really are interested in making an informed oppinion. There is alot of politics going on too

Wes Hall12:02:21

@UQXS91RT7 I agree with that in principle. Just important to bear in mind that there is also content that exposes some flaws. His views on religion for example, are a god awful mess in my opinion. The discussion between him and Matt Dillahunty exposes a lot of that, as does the Sam Harris one mentioned above. He's an interesting character, and not afraid to say things that put him in conflict with the zeitgeist... a valuable thing, but there is also a bit of "follower" / "hero worship", that surrounds him which gets a little nauseous.

Gulli12:02:51

I agree with the religieous aspect, as well as the hero worshipping, but one of those things he can not do much about. He's helped alot of people out though, one of whom I know personally. That person is now a much better person and interacts in a much better and positive way towards others around her. He isn't for everybody though, that is for sure.

Aleksander16:02:41

It's the question who you give your attention to and who you allow to be influenced by. I think it's a fallacy that you can watch a long talk and not be influenced by it, especially if you are not well prepared to critically analyse it by yourself and don't read/watch critiques from other viewpoints. It's the question whether you listen to a hugely popular internet figure with as you've said "religious following", "hero worshipping" and from what I've seen: real-life meetups dedicated to analysing his thought. Or do you listen to reservations from less powerful groups/minorities. People I trust have a lot of objections against him so I won't be spending trying to understand how he thinks. In terms of his dangerous influence: I don't know the names of "Sam Harris" and "Matt Dillahunty" but I know eg the name of "Milo Yiannopoulos" - explicitly far right figure - which the person we're talking about introduces to his followers. The name of the guy is often mentioned as important stepping stone in radicalisation of vulnerable people. And finally cutting off the people from the views of far right people seems to work: https://www.vice.com/en_us/article/bjbp9d/do-social-media-bans-work