Fork me on GitHub
#clojure-uk
<
2019-11-06
>
3Jane07:11:36

It’s not correct terminology unless you can pronounce it in correct characters ;P

3Jane07:11:25

Interestingly Toyota also do FP, only in Elixir. Maybe we should ask them how they manage those teams XD

mccraigmccraig09:11:07

if each of those 1500 μsvcs has its own cassandra keyspace in a single cassandra cluster then they are quite a long way outside cassandra's documented limits for # of active tables (a few hundred)

mccraigmccraig09:11:59

although i guess many of those tables may be small and low traffic, so it might be ok

Ben Hammond09:11:59

bad news == didn't listen

Ben Hammond09:11:24

poor Cassandra

Ben Hammond09:11:30

oh I only just read the wikipedia page > According to legend, Cassandra had dark brown curly hair and dark brown eyes, and though beautiful and clever, was considered insane

dharrigan09:11:02

We're moving away from Cassandra for most of our stuff

dharrigan09:11:19

PostgreSQL FTW!

Ben Hammond09:11:33

Keepin It Simple

dharrigan09:11:00

I have some serious luuurve for postgres

🙂 8
dotemacs09:11:40

Would you be interested in attending ClojureX if it still went ahead? Please fill out this form to gauge interest, thanks https://forms.gle/YCCBzPuUKxNnRT2p9

mccraigmccraig10:11:32

@dharrigan i like cassandra... there's lots of stuff it's not good for though. what prompted your shift ?

dharrigan10:11:12

In the end, we found that postgresql does what we need, plus it's far easier to query

dharrigan10:11:25

we also use redshift for terabytes of data

mccraigmccraig10:11:37

yeah, we're dumping from cassandra to redshift for analytics queries. postgresql window functions are a thing of wonder

dharrigan10:11:33

they are indeed 😉

Ben Hammond10:11:16

slight tangent; what's feeling here about database triggers in Postgres? Personally I dislike them and I'm trying to untangle a project that has used them in unexpectted places

mccraigmccraig10:11:41

i've always been a bit nervous of them, but i might use them. i suspect that used well they could be simplifying things, and they can probably dig you out of a hole too... but they are quite opaque and could lead to difficult to comprehend emergent behaviours

Ben Hammond10:11:36

I've had systems fall over because another team inserted a trigger (that then started to fail), aborting the entire transaction and that soured me on them

Ben Hammond10:11:53

hmm perhaps that scales out into how I feel about reactive programming generally

Ben Hammond10:11:28

I'd rather play 'Follow the leader' than 'Chinese Whispers' Easier to troubleshooot

dharrigan10:11:49

I use database triggers

dharrigan11:11:04

Only one mind, but still very useful

Ben Hammond11:11:39

why choose a trigger, rather that do it from the clojure code?

dharrigan11:11:48

saves a roundtrip on network

dharrigan11:11:02

I would have to do a select, pull back the results, evaluate it, then do an upsert

dharrigan11:11:34

whereas, with the tigger, it happens within the same transaction of the insert, and happens there and then on the instance

dharrigan11:11:00

and this trigger has to fire for every row

Ben Hammond11:11:22

for when INSERT... ON CONFLICT ... just doesn't cut it

dharrigan11:11:19

the tigger is doing a calculation

🐯 4
otfrom11:11:12

I've been burned too many times by code in a database

otfrom11:11:05

re:cassie - it is really good at what it is good at, but not good for things like adhoc analysis (but then you don't really want to do analysis on your operational db anyway)

💯 4
👍 4
jonpither11:11:02

We have a breakfast event on the 21st of Nov coming up in Canary Wharf, aimed at the FinTech audience https://www.linkedin.com/events/6592788975490998272 - please come along if you'd find it useful.

maleghast13:11:41

Guys / Girls - I am having a senior moment and a complete failure of Google-Fu at the same time. How do I run a REPL using Tools / Deps that has pulled in my dependencies..?

maleghast13:11:43

at the moment I run clj and I know that the dependencies are not coming down 'cos the REPL starts immediately without a download notice, but I have never used this library ever before.

danielneal13:11:44

should just be clj

danielneal13:11:56

is your deps file correct?

maleghast13:11:04

I think so...

dharrigan13:11:17

Try deleting the .cpcache folder

dharrigan13:11:23

and rerun the repl

maleghast13:11:41

{:paths ["src"]
 :deps
 {org.clojure/clojure {:mvn/version "1.10.0"}}
 {libpython-clj {:mvn/version "1.6-SNAPSHOT"}}
}

maleghast13:11:59

does that ^^ look well-formed?

danielneal13:11:09

ah the deps need to be in a single map

danielneal13:11:36

{:paths ["src"]
 :deps
 {org.clojure/clojure {:mvn/version "1.10.0"}
  libpython-clj {:mvn/version "1.6-SNAPSHOT"}}}

maleghast13:11:47

Oh shit... Thx - rubber ducking strikes another victory for time-savings, but also unmasks n00bs...

danielneal13:11:31

we’re all n00bs at toos.deps I think

thomas13:11:56

I am a professional n00b with 20 years of experience!

maleghast13:11:49

Fair point both of you 🙂

dharrigan13:11:51

The older I get, the less I know

dharrigan13:11:56

esp when it comes to Javascript frameworks

maleghast13:11:13

Now all I have to do is figure out why I can't get libpython-clj to initialize

maleghast13:11:28

(it is at least downloaded and I can require it in)

Ben Hammond14:11:00

can I lower the tone with a Postgres windowing question; I want to run somethin like

SELECT max(reg.id), reg.zone
FROM
...
GROUP BY reg.zone;
only I don't actually want the max value, I want 3rd from the top I can run
SELECT NTH_VALUE(reg.id, 3)
       OVER (PARTITION BY reg.zone ORDER BY reg.id DESC),
       reg.zone
FROM ...
but this gives me a bunch of duplicate answers, presumably due to the moving window I've not quite got my head around this windowing stuff, so am probably overlooking the obvious solution any suggestions?

Ben Hammond14:11:00

and I can abuse rank() to do something similiar; but that seems a bit inefficient too

SELECT * 
FROM (SELECT reg.zone, reg.id,
             rank() OVER (PARTITION BY reg.zone ORDER BY reg.id DESC)  as r
      FROM ...
    ) rr
WHERE rr.r = 3;

Ben Hammond14:11:44

similiarly

SELECT * FROM (
    SELECT DISTINCT NTH_VALUE(reg.id, 3)
           OVER (PARTITION BY reg.zone ORDER BY reg.id DESC) as threshold,
           reg.zone
    FROM ...)) rr
WHERE rr.threshold IS NOT NULL;
gets me to the right outcome, but feels clumsy

Ben Hammond14:11:12

maybe that's my best bet

mccraigmccraig14:11:22

it's been a while... but iirc NTH_VALUE will give you a result for each row (i've not used NTH_VALUE before, but i have used LEAD and LAG to great effect), whereas you want to filter out just a single row from each partition - so your approach using rank seems reasonable

👍 4
Ben Hammond14:11:10

ah right. thanks

dharrigan14:11:18

You could look to use the nth_value with frame clauses?

dharrigan14:11:25

Not used that myself

Ben Hammond14:11:46

SELECT NTH_VALUE(reg.id, 3)
       OVER (PARTITION BY reg.zone ORDER BY reg.id DESC RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING),
       reg.zone
FROM ...
gets me lots of duplicates, but I can just DISTINCT them

yogidevbear14:11:40

Anyone here heard from @guy recently? I haven't seen him online in a while and want to check that he's okay

dharrigan14:11:30

anyone here use ssh with certificate authentication, instead of authorized keys?

maleghast16:11:20

I've worked somewhere that did - the BBC - but I have no idea how it was implemented I just know that I installed my cert and I was good to go.

maleghast16:11:31

(not that I had a lot of access tbh)

Conor16:11:57

I also worked there! The BBC has an internal CA that issues the certs

Wes Hall17:11:49

@yogidevbear I'll drop him a facebook message if you like.

Wes Hall17:11:08

He should now feel the love

dharrigan17:11:52

I've tested out my own using certs rather than authorized keys, seems to be the way forward!

Wes Hall17:11:12

@yogidevbear He's OK. Got some personal stuff going on that's keeping him a bit occupied, but he's doing alright.

❤️ 4
yogidevbear17:11:27

Thanks Wes, really appreciate it 👍

👍 4
acron17:11:03

I missed the conversation about recruitment but if anyone here ever has a bad experience with a FunctionalWorks recruiter, let me know about it please :cop: