Morning.

morning

Bore da pawb

morning

Morning

morning

Morning

morning!

@agile_geek what do you think burned you out about it?

Morning

Morning All!

Quick question - anyone in here done the whole HTTP Upload -> S3 without__ a temp file on the disk?

I think that it can be done using Amazonica, but I am struggling to find a crib / example to work from... (My Google-Fu has been WEAK of late)

I did once

but the code is in a place I can't see it any more

😞

there's a way to do it with amazonica iirc

I did as well some time ago, but I don’t have the code either 😞

I think you want to use this - https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/PutObjectRequest.html#PutObjectRequest-java.lang.String-java.lang.String-java.io.InputStream-com.amazonaws.services.s3.model.ObjectMetadata-

ahhh

Whatever you want to upload needs to go into that InputStream

that looks familiar

I don’t remember what the amazonica interface into that looks like though

nah me neither

is amazonica the one that's super dense & basically the tests are the docs?

pretty much

👌

@maleghast i went with the tempfile for s3 uploads - iirc i had to implement less to get it to work without blocking my threads

morning

o/

Morning people!

I love coffee 😄

☕☕☕

well, if Clojure runs on Java, it is appropriate for Clojurians to run on coffee

works for me. 🙂

So do any of you kind folks go on certification courses or things like that?

They tend to be fairly useless in my experience

Perhaps I've not been on the right ones

I would not make a hiring decision based on someone's certification or lack thereof

its more for personal development rather than getting a job

Just read a book, yo

if that makes sense?

in that case I think it depends on what your learning style is more than anything.

and on what your accountability model is (some people need external accountability like a gym buddy or trainer, some can do it themselves, some can only do it if they’re NOT supposed to do it)

I remember reading an article about why MOOCs have not turned out to revolutionise education: they have a huge dropout rate, something like 90%

I did a 2 day Clojure course with Stuart Sierra before I’d written a single line of Clojure. I found it extremely useful. If you’re not doing it for the certification I think there are really good courses out there

Though none that come to the top of my head

I think I'd never look for certifications on a CV really

the only time I'd think "okay, that's good" is an AWS cert for a platform engineer

just because for some reason platform folks are v variable in quality

(well, no more than software engineers)

but maybe it's that the pool is smaller

(and in terms of personal experience, I’ve done internal company trainings in a giant corp, and certification with external company (inviqa iirc) and also (human) language courses, and in each case I found that I self-teach faster and more effectively, but benefit hugely from dense reference-like resources. Thus, I agree with “read a book”, if that includes “and play/write your own test code at the same time”)

Some of the workshops that get put on with conferences have looked interesting, though I’ve never been to any. For example - https://craft-conf.com/2018/workshops

and certs are very industry-specific; apparently if you want to do security they’re worth the money. Cisco used to be worth it as well, I don’t know about these days.

ok thanks all!

welcome!

back 😄

Thanks All, specifically @carr0t @mccraigmccraig @alex.lynham @firthh and @jasonbell for S3-based input... I had to do a conference call and then I was head-first in some research and then lunch and finally I remembered that I asked a question in here...

😆

@mccraigmccraig - When you went with tempfile on the disk in order to no block any threads, did you have a separate app / worker pushing the files to S3?

I;ve not found any training courses I've done to be hugely useful, but I think I've possibly not done them early enough

Normally I seem to pick something up, we work or I experiment with it for a few months on and off, then I/someone goes on a training course to see if we're missing everything/if we're doing it right

I've had little bits of training here and there that I found very helpful, but I know what you mean @carr0t

Which means often you're sat there for 95% of the course wishing they'd get onto more meaty stuff

Hehe

And yet at the same time there's other people there who are new to the (whatever), who are asking "Why are we doing X? What does Y mean?", and it often isn't covered so well in the course

@maleghast just looked it up - yes, we're using a manifold.deferred/future to do the transfer, which uses a separate thread

Even the Cisco courses I did, and sysadmin cert ones, were "enter these commands", but no real explanation of what the commands were for and why you were entering them. Which was fine for me, as I'd been doing net/sys adminning for a while, but meant that I didn't get as much out of it as I would have liked, and ended up helping other people understand what it all meant

@mccraigmccraig - Thanks, that's a paradigm I grok - I have a similar approach to doing database inserts off the back of async API calls.

it's not my favourite @maleghast - it's one of a very few places in our app where we aren't properly non-blocking, but doing it properly would be too much work for something that doesn't happen that often, so a thread it is

*nods*

training courses are often good (I've helped teach some and learned clojure from Stu Halloway many moons ago, and having him to ask questions was really useful)

certification is a different thing. Sometimes even the "useless" ones are handy as they can be used as a filter by some bits of the org that aren't as clueful, but can help you get into a team that does have a clue

(I'm thinking things like scrum certs)

some of the scrum trainers will actually give you some good info in addition to the bit you need for the certification

Yeah, I found doing the Juxt Advanced Clojure with @malcolmsparks and @jonpither last year was really helpful.

I was 2 years from having done any "Clojure for money" and it really helped me get back into the swing of things, and pick up on some new (to me) ideas.

one of my questions to Stu H (after being frustrated by common lisp, emacs lisp and scheme) was "will map and reduce be the way to actually write production code that is suitably performant" and his answer was "yes"

I think reducers, transducers and core.async show that to be true

and I wouldn't have been able to get that from a book

Yeah, that is the sort of thing that you need from "the Horse's mouth"...

what do you think horseshoes are?

you don't get to leave now @thomas

Omega's?

only in the end

in that case I don't know I'm 'fraid.

(  (
(  (

ooohhhhh..... parens!

is a horseshoe just a (partial reduce +)?

obviously they need to turn around to close the statements

lol

😂

@alex.lynham that use of partial really hurts me given the arities + takes. 😉

haha I partially (see what I did there) wanted to troll

uauahghgauahghgughaghhg

the 'joke' also I guess wouldn't work with (reduce + <horse?>)

I overreached with this one I think

it was tenuous at best

quite an achievement... do we finally talk about Clojure... do we somehow manage to get back to other things in no time.

there’s a reason they call it a stable release.

:drum_with_drumsticks:

¯\(ツ)/¯

wait hang on, I was thinking of sigma, not omega

ignore me

I don't have a lousy syntax joke about the end of a set

so... game, set and match to @lady3janepl and me?

solid jokes folks 10/10

😱

@thomas I hang my head in shame

the bell of shame

i know it well

@guy <<the bell of shame>> Now what have i done?

🍩

if you have some reference data in a file that you use, what is your preferred way to make it available in your application?

say a csv file that you turn into a lookup table that gets used by lots of other functions

how long have you got to complete the task?

👀

or maybe i read that wrong

Do you mean, where do i store the csv? like sticking the csv in the resources folder?

@otfrom depends on how it's going to get used - all yapster's bulk imports get turned into records in a bunch of append-only import tables, basically a direct representation of the import source data in a common scannable format

after that those import tables get joined with the tables from the live database and used to generate changes

mostly I'm trying to avoid doing it in a def as if I get a problem with the file, it will blow up during compilation

the sizes are very small

so no need to put it into a database, and again it is a batch process so I only need it in memory for the script to run

memoize it ?

that was my thought, but wanted to get other opinions

turn the calls from foo into (foo), but have the function memoized

so that it wasn't reloading every time

delay?

but at least then if it goes bang it happens at run time rather than compile time (and my exceptions would be easier to read and the ns would actually load)

@dominicm not used delay before

delay is more appropriate than memoize here i think

that looks pretty good actually

delay

thx @dominicm!

either way don't do it at compile time

yup

makes startup a pain too (when the files get biggish or the processing long)

future is tempting too

ah, future attempts to do things immediately, delay waits until it is dereferenced

delay is what I want

Interesting. Not come across delay before

Yeah

mmm... tried combining some futures with some JNI and that... didn't go well.

I do like being able to core dump the jvm tho 😄

pretty sure the JNI is trying to be "clever" about how it manages file resources and not doing it very well

Hi folks

I’ve probably asked this before

it's been long time... is this helpful @otfrom http://www.kfu.com/~nsayer/Java/jni-filedesc.html ?

but why does (conj nil nil) produce (nil)

@guy 'cos you are adding a nil value to an empty list, giving you a list with one nil element

so nil is an empty list?

😅

@mccraigmccraig the resource handling looks to be pretty buried in the library. It is doing some kind of reference counting. I'm just gonna run it single threaded

@guy squintily

cos (seq nil) is nil ye? so it counts as empty?

(conj nil 1)
(1)

conjing on to a nil gives you a list with the 2nd arg in it

but why

the first nil is the collection you are conjing onto

but how is a nil a collection

👀

nil isn't a collection

from the docs

Added in 1.0
  conj[oin]. Returns a new collection with the xs
    'added'. (conj nil item) returns (item).  The 'addition' may
    happen at different 'places' depending on the concrete type.

oh right

@guy 'cos it's always been that way since way before clojure was a twinkle in rh's eye

😰

ok thanks!

more nil punning sugar I think

what is nil punning sorry

is that like returning nil when something doesn’t return what you want it to?

the java that implements conj is here: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java#L667-L670

static public IPersistentCollection conj(IPersistentCollection coll, Object x){
	if(coll == null)
		return new PersistentList(x);
	return coll.cons(x);
}

I think this is a Lispism

thanks @otfrom !

it is

(in that nil is the same as an empty list *in Lisp)

some history here: https://lispcast.com/nil-punning/

it’s not that it’s a collection, it’s literally defined as an equivalence

oh, that’s what it’s called 😄

oh great thanks @otfrom

@lady3janepl except that an empty list doesn't equal nil

(= nil '())
false

😕

(but yeah)

yeah

it’s seqable? and (seq nil) returns nil because they made it that way

(one interesting thing I found is that you can extend-protocol to nil)

(but then if you have a more than one whatever-type implements the protocol, it’s not possible to fluently cast between nil and “empty type X”. I’m guessing that’s why there’s no equivalency in Clojure - at the base level things just aren’t lists anymore?)

oic

nil represents an abstract concept of nothing. what happens if you conj 1 to nothing? well, conj returns a collection, so it must be a collection containing 1, hence (1). similarly if you do (map inc nil); map returns a sequence - what's the sequence equivalent of nothing? an empty sequence, hence ()

nil not being the same as () is one of the differences Clojure has from other lisps, documented here: https://clojure.org/reference/lisps

interesting it still chooses list as the type to convert nil to 🙂

(by which I mean: if you conj to a vector, you’ll get a vector; if you conj to nil, you get a list)

nil-punning is the devil's work