Fork me on GitHub
#clojure
<
2019-01-24
>
sova-soars-the-sora00:01:12

(take 100 (sort-by :timestamp > @nf-chat-messages)) what am i doing wrong to get a null pointer exception o.o

seancorfield00:01:32

@sova sounds like :timestamp is missing from one of those messages…

seancorfield00:01:55

try (every? :timestamp @nf-chat-messages) and see whether it’s truthy — I bet it’s not.

dtsiedel03:01:23

Hi all, By my understanding of https://clojure.org/reference/reader, .1 should not be a valid identifier, since it is not an available symbol ("Symbols beginning or ending with '.' are reserved by Clojure"). And yet in my REPL:

user=> .1

CompilerException java.lang.RuntimeException: Unable to resolve symbol: .1 in this context, compiling:(null:0:0) 
user=> (def .1 0)
#'user/.1
user=> .1
0

dtsiedel03:01:06

Am I misunderstanding the description of a symbol?

hiredman03:01:15

"reserved" means you shouldn't use them, it doesn't mean clojure always cares if you do

alexmiller04:01:42

in this case, I definitely would not

dtsiedel04:01:19

Ah I see, I interpreted "reserved" as "invalid as an identifier", but that makes sense. Thanks!

dtsiedel04:01:40

And yes, I wouldn't define .1 as anything unless you're in a Clojure obfuscation competition 😅

noprompt05:01:13

Seems like the language wording could just be improved to say exactly what @hiredman is said and leave no interpretation up to the reader.

seancorfield05:01:11

@noprompt It’s common for languages to define a set of “reserved by <the language>” symbols — without giving more meaning than that.

seancorfield05:01:23

Java does it, C/C++ both do it…

noprompt05:01:54

But in those cases the symbols are reserved because they’re keywords.

noprompt05:01:57

The real reason why you shouldn’t define a symbol that leads with a dot in Clojure has everything to do with the how the reader reads list forms that have a symbol which leads with a dot.

noprompt05:01:59

(def .a (fn [] 1))

.a
;; => 
#function[scratch/-DOT-a]

(.a)
;; Malformed member expression, expecting (.member target ...)

seancorfield05:01:03

No, in many languages, certain characters in certain positions are “reserved” — and may or may not work, and may change their meaning in the future.

noprompt05:01:36

Okay, sure, I’ll buy that. JavaScript is like that. But my point is regardless of who’s saying it, it’s still unclear.

seancorfield05:01:04

It’s got a very specific meaning in language definitions tho’…

seancorfield05:01:39

It’s like in Standards, where unspecified/undefined/implementation-defined terms are used.

noprompt05:01:47

The fact we’re having this discussion proves there’s a misunderstanding and that the language is not clear because it can be misinterpreted.

seancorfield05:01:48

(and shall/should/may/must)

noprompt05:01:13

Right. But! In those cases the terms are defined up front.

noprompt05:01:28

And they’re defined up front for that very reason.

seancorfield05:01:07

Well, Clojure doesn’t have a “standard”, even tho’ it uses common standard words in its description.

noprompt05:01:53

There’s a big difference between well defined and, well, defined.

seancorfield05:01:06

“Well, actually” 🙂

noprompt05:01:31

It doesn’t seem like defining terms up front would be a bad idea though. It’s useful.

seancorfield05:01:51

True, Clojure could add a Glossary… if it doesn’t already have one…

noprompt05:01:04

It’s a problem in software engineering in general.

noprompt05:01:48

Having to be clear is no fun. 🙂

seancorfield05:01:23

Having spent eight years on the ANSI C++ Standards Committee, I can assure you that “being clear” can seem overrated 🙂

noprompt05:01:26

Back to the case with the reader though; it would be helpful to explain why it’s reserved.

seancorfield05:01:54

All it could say is “reserved for future use”…

noprompt05:01:02

Or, at least, in that context, what “reserved” means.

noprompt05:01:02

The example I provided above is a good illustration.

seancorfield05:01:05

That’s what “reserved” means.

seancorfield05:01:22

It may work now, it may not. Even if it works now, it may not work in the future.

noprompt05:01:36

I’m sure it has it’s drawback but we shouldn’t throw the baby out with the bath water.

seancorfield05:01:56

It may partially work (as the .1 examples above show) or not at all or it may launch missiles 🙂

noprompt05:01:56

Specs and clear language are really useful.

seancorfield05:01:35

Would you be happier if it said “symbols beginning or ending with . have undefined behavior”?

noprompt05:01:04

No. Because that would be lying. It has precisely defined behavior.

noprompt05:01:33

It’s precisely defined in the reader.

seancorfield05:01:10

I don’t know that you could accurately write down the situations under which .a would behave in every possible way tho’…

seancorfield05:01:35

(and I don’t know that such a description would be useful even if it were accurate)

noprompt05:01:10

Well, I suppose, if Rich decided that it’d be worth checking if the symbol can be resolved before treating it as a method call… we could say something along the lines of “this is it’s definition now”.

seancorfield05:01:32

And the problem with writing it down is that then if the behavior changes accidentally in some situation in the future, the description may no longer be accurate — whereas “reserved” imparts no specific meaning and would not be rendered inaccurate by such changes.

seancorfield05:01:27

Clojure has a long history of only specifying partial behaviors — and leaving implicitly undefined a lot of behavior. In order to allow the implementation to change/improve and only be bound by the explicitly defined partial behaviors.

seancorfield05:01:33

clojure.set is the classic example — the functions are defined for inputs that are sets. If you give them non-sets, you still get results, but they’re not specified by the docstrings — and could change whenever the implementation gets improved.

seancorfield05:01:08

There are extremely good reasons in language design — and language specification — for not specifying the behavior of a certain set of constructs.

noprompt05:01:42

I’m not sure if that’s a good argument. Maybe I’m reading this wrong but are you suggesting deliberate usage of vague language?

seancorfield05:01:10

No, I’m saying undefined or unspecified behavior is a valuable — and extremely common — part of language specification.

seancorfield05:01:25

Those are not “vague” terms.

noprompt05:01:47

Eh, I mean you could say that but in this case I don’t think there’s a lot of room for symbols that lead with a dot to do much else besides be resolved to vars.

noprompt05:01:13

I do think it’s better than “reserved”, however, I’ll give you that.

noprompt05:01:07

Regarding intentional partial behavior, I think it’s fine up to not being open about it. That clojure.set does not tell someone up front that, hey, if you give me non-set stuff I can’t promise you it’s gonna work, is borderline dishonest.

seancorfield06:01:10

I think we’re just going to disagree.

noprompt06:01:45

I’m not sure we’re in disagreement. I’m agreeing with you that “undefined” is superior to “reserved”.

seancorfield06:01:06

For me, with a lot of background in language design and specification, what the Clojure reader page says about .a / a. etc is perfectly clear and reasonable.

seancorfield06:01:17

The same goes for clojure.set.

seancorfield06:01:27

It defines the behavior for specific inputs.

seancorfield06:01:38

It therefore does not define the behavior for other inputs.

noprompt06:01:57

Sean, I have a lot of experience with language design and specification as well.

noprompt06:01:02

But, Sean, there’s no disclaimer on the namespace that says “GIGO for non-maps”.

seancorfield06:01:14

There does not need to be.

noprompt06:01:20

And that’s the problem.

noprompt06:01:31

Many people think there should be such a disclaimer

seancorfield06:01:43

Many people are wrong 🙂

noprompt06:01:52

I’ve been working with Clojure personally and professional for nearly 6 years and I’m one of those people.

noprompt06:01:15

I’ve seen people burned by that, by merge, and countless other so-call “GIGO” functions,

noprompt06:01:25

it’s punishment.

seancorfield06:01:42

This is why I said we’re just going to disagree.

seancorfield06:01:23

If the docstring says “Given A, the function does B / produces C” then you cannot expect any given behavior if you give it “non-A” as input.

noprompt06:01:23

What are you disagreeing with?

seancorfield06:01:21

Now, if you give it A and you don’t get C, then that’s a bug. But if you give it X and get C, that’s should not be unexpected.

seancorfield06:01:36

If you give it X and get Y, that should not be unexpected either.

noprompt06:01:38

It’s still not clear to me why omitting the fact the functions will accept non-set input and have undefined behavior.

noprompt06:01:58

We both agree that’s useful language, no?

seancorfield06:01:27

It’s superfluous. If the behavior is (only) defined for set input then it is implicitly not defined for non-set input.

seancorfield06:01:31

Given that Rich and several other Clojure/core folks seem to abide by that line of thinking — and it’s common across many language definitions (both ANSI and otherwise) — I’m not sure why folks think the docstring will (or indeed should) change…?

seancorfield06:01:12

Rich, in particular, is against over-specification — he’s said as much many times in his talks.

Mattias06:01:29

This is fascinating. Is there a longer text anywhere on the design choice to have easily accessible undefined behaviors to such an extent in the language?

noprompt06:01:30

Well you can’t be Clojure/core if you disagree with Rich.

seancorfield06:01:12

(I’m certainly not Clojure/core but given my background I tend to agree with him on a lot of stuff — not everything but a lot 🙂 )

noprompt06:01:02

And it’s not “over-specification” it’s just being clear about what you’re leaving operations on unexpected inputs undefined.

noprompt06:01:41

You can still do the whole GIGO thing while sparing everyone that’s ever used and will use Clojure the cost of having that WTF? conversation that inevitably crops up when “weird stuff” happens like mergeing two vectors.

seancorfield06:01:16

Like I say, we’re just going to disagree on this. I think it’s perfectly clear.

seancorfield06:01:51

@mattias504 I had hoped it might be referenced here https://gist.github.com/reborg/dc8b0c96c397a56668905e2767fd697f but it seems not (at least, not yet).

seancorfield06:01:16

Perhaps this should be added to https://clojure.org/guides/faq somewhere? But I bet people would still complain that the docstrings didn’t also include that same text… 😐

noprompt06:01:18

It’s par for the course for folks like me who have a differing view than that of Clojure/core et all to be marginalized.

noprompt06:01:10

We’re entitled!

seancorfield06:01:10

You’re not being marginalized. We’re just disagreeing.

noprompt06:01:08

I’m fairly sure that if I submitted a patch to clojure.set which added to the docstring a disclaimer about undefined behavior for non-inputs it’d be given a pass.

noprompt06:01:17

Could be speculating though.

seancorfield06:01:25

Correct. Because Rich has explained his stance on that several times.

seancorfield06:01:45

He is under no obligation to accept every patch nor even explain why he won’t.

noprompt06:01:54

Well, yeah, it’s his language.

seancorfield06:01:59

In that particular case, he has explained repeatedly why he won’t.

seancorfield06:01:51

I’ve asked for stuff in the past and he’s shot me down too. I don’t feel marginalized. I just accept he and I have different viewpoints on some things.

noprompt06:01:52

It doesn’t seem like a reason though. Like, there’s nothing, literally nothing wrong with explaining that clojure.set has undefined behavior in the docstring. Zero.

seancorfield06:01:20

I’ve explained above why adding that sort of stuff is pointless (from the project’s p.o.v.).

seancorfield06:01:26

You just don’t agree.

seancorfield06:01:58

And that’s fine. We don’t all have to agree. But sometimes we need to accept that folks don’t agree.

noprompt06:01:07

I don’t because I’ve been a party to so much buggy Clojure code and long debugging sessions.

noprompt06:01:40

If it was a one of thing that happened once in a while, I’d be more likely to agree with you.

seancorfield06:01:39

I’ve had Clojure in production for almost eight years now. I’ve hardly ever run across “buggy Clojure code” in Clojure itself, nor much in most libraries to be honest. Bugs in my code, sure. And very few long debugging sessions. But, yeah, sometimes I chase my tail for longer than I’d like — because I did something wrong.

seancorfield06:01:05

We’ve run prerelease builds of Clojure in production nearly all that time, from an alpha of 1.3 up to date.

noprompt06:01:21

Well, Sean, that’s a nice anecdote and I’m happy for you but at virtually every company I’ve worked at that’s used Clojure I’ve seen the issues that people bring up.

noprompt06:01:57

The stance to not put in a disclaimer because “it’s pointless” is void of any kind of compassion. But, again, we’re not entitled to compassion.

noprompt06:01:00

I remember having similar discussions about the horrible error messages years ago and being effectively told that it was my problem.

noprompt06:01:15

We’re getting better error messages now so maybe, given a few more years, we might see some willingness to at least hear out these people (like me) who’d like to see some more transparency in the docstrings.

flefik06:01:45

idk, it’s pretty standard with ‘undocumented’ undefined behaviour. c++ comes to mind as notorious for this. you document your domains and co-domains, not everything that’s unsupported. Adding a docstring just add noise. For examples and other info refer to some 3rd party resource. Like http://clojuredocs.org for example

noprompt06:01:23

So this phrase “it’s standard” seems like it’s a synonym for “that’s tradition”.

noprompt06:01:43

Kinda like bad error messages. They’ve always sucked and they’re gonna stay that way.

seancorfield06:01:43

Well, in C++’s case, it’s literally “Standard” — as in ANSI/ISO 🙂

flefik06:01:03

well there are plenty of good reasons why it’s standard

flefik06:01:09

checking inputs cost cycles for one

noprompt06:01:23

Yeah, I’m not saying there should be checks.

noprompt06:01:44

I said the GIGO (garbage in garbage out) is fine.

noprompt06:01:03

I’m on board with domain/codomain as well.

seancorfield06:01:27

OK, here’s a question then

clojure.core/+
([] [x] [x y] [x y & more])
  Returns the sum of nums. (+) returns 0. Does not auto-promote
  longs, will throw on overflow. See also: +'
The docstring does not define the behavior of (+ 'a :b "c") — do you think it should?

noprompt06:01:58

That thing throws if you give it anything other than numbers.

noprompt06:01:10

So, right there, you’re analogy is missing the point.

seancorfield06:01:17

The docstring does not say that.

noprompt06:01:24

Set will take non-set inputs and not throw.

seancorfield06:01:33

It could do anything if you give it non-numbers.

seancorfield06:01:42

That is exactly my point.

seancorfield06:01:11

You “know” that giving + anything but numbers will throw an exception — but it doesn’t say that.

seancorfield06:01:39

It could quite happily give you any result — or spin off into infinity — and still satisfy its docstring.

noprompt06:01:43

No, no, no. That is not analogous.

seancorfield06:01:57

Again then, we’re going to disagree.

noprompt06:01:08

clojure.core/merge and clojure.set/union would be good analogies.

noprompt06:01:34

Both will accept inputs that have undefined behavior for their outputs and not throw.

noprompt06:01:53

+ on the other can surely receive invalid inputs but will throw given them.

seancorfield06:01:14

But their docstrings do not specify the behavior for non-maps / non-sets. That’s exactly the same as + not specifying the behavior for non-nums.

noprompt06:01:00

My point is that because they do not throw, those functions should explain they have undefined behavior for unexpected input types.

noprompt06:01:14

+ doesn’t need to because it will throw.

noprompt06:01:29

The error will tell you that you did something wrong.

flefik06:01:41

why would you not assume everything has undefined behaviour for unexpected input?

👍 1
seancorfield06:01:17

(! 1374)-> node
> 1 + "s"
'1s'
> 1 + []
'1'
> [] + {}
'[object Object]'
> 
I don’t even know where to begin to explain that…

noprompt06:01:47

Well, most people would assume the function would throw for bad input but that’s not “the way” for the reasons Sean mentioned above.

seancorfield06:01:20

“most people would assume the function would throw for bad input” — disagree.

seancorfield06:01:25

Anecdotal at best.

seancorfield06:01:03

@cfeckardt Yeah, I know… JavaScript has a lot of weird behavior 🙂

flefik06:01:05

your examples make sense, that one does not

noprompt06:01:16

It also has a spec that tells you about it too.

seancorfield06:01:17

(for the curious {} + [] produces 0)

😭 2
noprompt06:01:31

jsfuck ftw! 🙂

noprompt06:01:41

Yep. Tells you exactly what it’s gonna do.

seancorfield06:01:15

And you think that would be better than Clojure’s docstring for +? 🙂

noprompt06:01:32

Ah but, damn, if they had just decide not to do that they could have reserved the right to do whatever they wanted!

seancorfield06:01:58

To be honest, reading that ECMA spec for +, I’m not sure it says what should happen when either operand is neither a number or a string.

seancorfield06:01:15

It’s pretty clear about num + num and str + str tho’…

flefik06:01:57

well [] is analogous to empty string

flefik06:01:14

so 1 + '' == '1'

seancorfield06:01:21

(in particular, it doesn’t give any indication that swapping the left and right args gives different behavior)

flefik06:01:02

and {} cant evaluate to a number, so it’s cast to it’s string representation which is ‘[Object object]’

Mattias06:01:01

Hmm. Undefined behavior is undefined by definition, ie the doc string definition. It is not immediately obvious why that information is kept as programmatically unactionable text. Had it been structured the reader/compiler/runtime could have used the same information (“-flagtomakeundefinedcallsthrowexceptions”). Is it a strength to keep the knowledge of what is undefined as docstrings (that are practically required reading).

seancorfield06:01:34

@cfeckardt The spec says GetValue -> ToPrimitive on both args and if either are strings at that point, then convert both to strings and concatenate them. Else if neither primitive type is string, convert them both to numbers and add them.

flefik06:01:45

yeah but that does not explain the difference between {} + [] and [] + {}

flefik06:01:33

it must mean that the left argument is evaluated first

seancorfield07:01:59

It seems to be down to the difference between ToNumber [] and ToNumber {} — but right now I am not convinced the spec actually defines the result and so we’re back in unspecified behavior territory (which was my point).

Mattias06:01:45

Really fascinated by the topic, reading your discussion with interest. Thanks for the thoughtful (and patient) explanations 😀

seancorfield06:01:36

…and ToNumber {} is well-defined (bizarrely) in 7.1.3

noprompt06:01:05

I’ll rephrase my previous remark about “most people”’s expectation to throw as “many people”. Editing that now would be… weird.

noprompt07:01:01

I’d also accept a clojure spec for clojure.set functions and clojure.core/merge.

andy.fingerhut16:01:06

Some unofficial places where these things are expanded on a bit: http://ClojureDocs.org has examples of things that can go wrong if you give non-set args to clojure.set functions, warning about them in English

andy.fingerhut16:01:47

This 3rd party library https://github.com/jafingerhut/funjible will throw exceptions if you pass non-set args to clojure.set functions, and has been designed as a fairly easy drop-in replacement for clojure.set functions.

andy.fingerhut16:01:14

@U04V15CAJ is working on specs for many Clojure functions, including those in clojure.set, over which there are quite a few fine points on what types they should be spec'd to accept: https://github.com/borkdude/speculative as for example on this issue: https://github.com/borkdude/speculative/issues/161

andy.fingerhut16:01:52

I realize that these do not touch on your questions about official doc strings. There are Clojure JIRA issues about clojure.set functions that are open, and some closed.

borkdude16:01:15

There’s also an issue with a proposal to coerce to sets in set functions.

andy.fingerhut16:01:20

There are a few very vocal people working on things like this. 11 downloads over 7 years for funjible library, so how much do people really care? On average, it seems not a lot.

borkdude16:01:28

For fun, here are some examples of 4clojure functions that rely on set functions doing the right thing with their input. I changed them to conform with the specs. https://github.com/borkdude/coal-mine/commit/5c9a009dc326b0ab598a419777e71a04ab000ccd

andy.fingerhut16:01:58

Those were solutions written by people using 4clojure to solve its problems, or code that was used to implement the 4clojure site? I am guessing the former.

borkdude16:01:18

The former, yes. And they work for their problem input and output

borkdude16:01:33

But we use this to verify the specs in speculative, so I’m fixing it to not see the same errors every time

noprompt16:01:50

The small number of funjible downloads alone can’t be used to infer how much people care. For example, I care about the issue yet this is the first time I’ve heard about funjible. Today is also the first time I’ve heard about speculative; I’m interested. 🙂 @U0CMVHBL2 I noticed you’ve submitted a patch to add some fdefs for clojure.set. It looks like there’s some potential to get it merged. It would be great to see that happen.

borkdude16:01:49

Clojure set fdefs are already in speculative

borkdude16:01:00

(Typing from mobile )

noprompt16:01:26

@U04V15CAJ Yes, I noticed that. 🙂

noprompt16:01:38

(Not that you were typing from mobile.)

borkdude17:01:15

oh, you did not mean a patch for speculative but clojure.core?

noprompt17:01:50

@U04V15CAJ Yeah. Rereading my reply I can see the ambiguity.

andy.fingerhut17:01:43

I am sharing this info with you @noprompt since you seem to be one of the few who care 🙂

borkdude17:01:06

A cult within a cult

noprompt17:01:14

It’s appreciated, for sure. 🙂

andy.fingerhut17:01:03

I could advertise the existence of funjible once per month, even if it doesn't change, but I think the Clojure Google group and/or announcements channel on Slack would quickly get tired of the repetition.

andy.fingerhut17:01:43

The existence of popular libraries with lots of common use cases are advertised by those other than their authors 🙂

borkdude17:01:28

it’s surprising how infrequently I’ve been bitten by the current behavior of clojure.set

borkdude17:01:59

given the examples of what could go wrong

noprompt17:01:30

Once you’re burned you don’t forget.

noprompt17:01:57

Like merge. A few weeks ago I helped debug a situation where (merge-with merge ,,,) caused a bug that took, let’s just say, a non-zero amount of time to get to the bottom of.

borkdude17:01:23

speculative chooses to only allow maps as args of merge.

borkdude17:01:43

don’t know if that issue had to do with that

noprompt17:01:17

That aligns with my intuitions and I think others’ as well. I’ve never heard anyone say they wanted to use merge as a substitute for conj.

borkdude17:01:40

that’s what I see a lot in the 4clojure solutions I test speculative with. merge -> conj

borkdude17:01:31

it isn’t based on intuition though. first we had associative? but if you look closely at the implementation, a {} is substituted for a non-truthy first argument

noprompt17:01:50

I realize it’s not based on intuition. What I meant was that people I’ve worked with intuitively expect merge to work with maps alone and that, one way or another, for an exception to occur in other circumstances.

noprompt17:01:07

Vectors are associative? 😄

andy.fingerhut18:01:54

I agree that many people would wish an exception to be thrown in situations where you call a function with arguments it is not designed to do something reasonable with, and I am sympathetic with such wishes. That doesn't mean Clojure core implementation is going to change to align with those wishes. The options we have, that I know about, anyway, are what I've mentioned above: alternate docs, spec, and things like funjible. That and perhaps performance measurements of approaches like funjible takes, on serious code bases, that may (or may not) convince Clojure core folks that the costs of the extra checks are acceptable.

❤️ 3
flefik07:01:37

gotta go to bed it’s late here 😞 goodnight

seancorfield07:01:29

Yeah, my partner’s already gone to bed so I should probably join them…

Mattias07:01:47

(I’m just about trying to get kids off to school 😅)

noprompt07:01:55

Fun having this discussion with you @seancorfield! 🙂

seancorfield07:01:35

Heh, I’m always up for discussions about language semantics and standards etc — it’s what I’ve been doing on and off since the early 80's 🙂

noprompt07:01:03

Maybe one day I’ll be in your shoes saying what you’re saying.

seancorfield07:01:35

Then we’d have nothing to “argue” about 🙂

noprompt07:01:52

I secretly hope not because that’ll mean we’re still telling computers what to do with text. 😭

Lennart Buit07:01:20

meh, rather have computers be told what to do with text than with speech

seancorfield07:01:27

Could be worse. Could be switches, setting 0's and 1's… I remember doing that in the early 80's! Argh!

Lennart Buit07:01:43

I am pretty sure I will never be comfortable talking to my computer and the thing actually talking back :’)

noprompt07:01:56

I’m going to be sad when I teach my son about the terminal. “Son, this thing’s been around since before I was born.”

noprompt07:01:36

@lennart.buit I regifted an Echo Dot because it’s just too creepy for me.

Lennart Buit07:01:08

Yeah, yelling at my computer because my code is crap is one direction only

seancorfield07:01:16

@lennart.buit I must confess that I’m only very slowly getting used to talking to my phone… I still resort to thumb typing, even when just telling it stuff would be faster/easier!

seancorfield07:01:25

👋:skin-tone-2: 🌃 😴

Lennart Buit07:01:50

while here in mainland Europe, its just starting to become light :’)

Christian Johansen08:01:58

is there a way to permanently disable the namespace printing behavior? e.g. set *print-namespace-maps* to false for the whole process

Christian Johansen08:01:38

specifically I'm trying to avoid this feature in kaocha test runs, and using set! just gives me "Can't set! *print-namespace-maps* from non-binding thread"

Christian Johansen09:01:42

to answer my own question, I ended up creating my own testing macro that generates a call to clojure.test/testing that wraps the body in a binding. alter-var-root didn't seem to work either

lilactown15:01:41

is anyone using cognitect’s aws-api library?

lilactown15:01:05

I’m wondering how I’m supposed to clean up / maintain my AWS client after I have created it

lilactown15:01:27

ex I found that just doing

(def s3 (aws/client {:api :s3}))
apparently creates some sort of long-lived connection that dies after awhile. so my service was working just fine after a deploy, but would eventually stop working 😕

lilactown15:01:41

I could just create a new client every invocation, but I’m now afraid of creating too many of these clients if they do in fact create a connection that hangs around

lilactown15:01:41

there doesn’t seem to be any public way of stopping a client after use, either. There’s a stop function in the cognitect.aws.client namespace, but that’s documented as "Impl, don't call directly."

ghadi17:01:44

@lilactown I couldn't reproduce that

ghadi17:01:52

what do you mean by "long-lived connection"?

ghadi17:01:00

do you see an open socket?

lilactown18:01:56

I’m honestly not sure. all I know is that my Datomic Ions can’t connect to S3 after a few hours unless I redeploy

lilactown18:01:12

it’s hard for me to tell what is actually happening. I got some advice in #datomic

hiredman18:01:18

sounds like security credentials expiring

lilactown18:01:31

☝️ that makes more sense!

hiredman18:01:01

depending on how things are deployed and where you are getting credentials from aws, automatically rotates them after some time period (I think most libraries automatically roll over on to the new ones)

hmaurer18:01:04

Is there a shorter/nicer way to check if a map has no non-nil keys than (empty? (into {} (filter x)))?

dpsutton18:01:18

(every? (comp some? val) {:a :b :c nil})

noisesmith18:01:24

(empty? (filter some? (vals x)) for non-nil values

spieden18:01:59

“has no non-nil keys” — so check that every value is nil? (every? nil? (vals my-map))

borkdude18:01:45

if you want to check the keys, you need keys 😉

noisesmith18:01:09

for that you just need (not= [nil] (keys m))

hmaurer18:01:14

@spieden ah yes that’s what I meant; (every? nil? (vals my-map)) is neat !

John Miller19:01:22

Anybody using the Cognitect AWS library with CloudSearch (the cloudsearchdomain service specifically)? You have to supply an endpoint somewhere, but I can’t figure out how. In the JS library, it is part of the constructor - https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/CloudSearchDomain.html#constructor-property. So I would expect it to be sent in the aws/client call.

ghadi19:01:23

We have a plan for tackling custom endpoints @jmiller . I don't believe it's possible right now.

😞 1
John Miller19:01:23

Thanks. I’ll go back to the old way and keep my eye out for updates.

Lennart Buit21:01:59

There is no such thing as strengthening your specs, right. Lets say that I have a map with ::some-ns/a ::some-ns/b and ::some-ns/c keys that all functions promise to be int?, but some promise to be pos-int?

Lennart Buit21:01:09

(obvious simplified example)

Lennart Buit21:01:32

I basically want to say something like, this function adheres to the weak contract (`int?`), but as an added bonus, also to this stronger one (`pos-int?`) ^^

john21:01:16

doesn't int? give you the added bonus of pos-int??

Lennart Buit21:01:21

sorry? I don’t quite understand, -5 is int? but not pos-int?. So I would spec something like this right: (s/def ::some-ns/a int?) with (s/def ::some-ns/my-map (s/keys [::some-ns/a])). Now I appear to have no way to write a spec for the stronger guarantee for the same namespaced key.

hiredman21:01:56

you shouldn't do that

hiredman21:01:17

it sounds like you want to add context

Lennart Buit21:01:20

why tho, I am promising you strictly more, I am still adhering to the weaker contract

hiredman21:01:44

say "oh, in this context this key is X, and this context the key is Y"

hiredman21:01:13

spec removes context, it is all global

hiredman21:01:31

the point of spec is to be able to say "for all maps, with this key, the key means X"

hiredman21:01:43

so if you want it to mean Y, add a new key to your map and spec it to be Y

hiredman21:01:31

if you want to say "this key is X and Y" s/and

Lennart Buit21:01:48

yeah, but Y implies X in this case. Every pos-int? is also an int?.

hiredman21:01:13

then just say "this key is X"

john21:01:41

said another way, you want to merge two specs and be allowed to choose the weaker or stronger one?

hiredman21:01:58

the issue isn't actually anything to do about "stronger" whatever

hiredman21:01:12

the issue is you are trying to define the same key to different meanings in different contexts

alexmiller21:01:24

you can s/and in stronger criteria in some contexts

hiredman21:01:32

in one context you want it to mean int? but in another context you want pos-int?

alexmiller21:01:40

but it’s admittedly pretty annoying to do so under s/keys

Lennart Buit21:01:42

its for function contracts

Lennart Buit21:01:02

I have some functions that have stronger guarantees than strictly required by the “global” map spec, so to say

alexmiller21:01:04

you could make a loose contract and a strict contract and combine with s/merge (then it has to satisfy both)

alexmiller21:01:26

but to some degree this is the kind of thing the select stuff will add in spec 2

hiredman21:01:31

you can't do that for the same key though

alexmiller21:01:33

that Rich talked about in his keynote

Lennart Buit21:01:42

yeah I watched it, was a bit afraid of that

Lennart Buit21:01:54

or --- not :’). Because you guys are obviously thinking about it ^^

hiredman21:01:13

you can always and on arbitrary predicates that are not s/keys based, that can do whatever

john21:01:44

that's true

hiredman21:01:51

(s/and (s/keys ...) #(pos-int? (get % :foo)))

Lennart Buit21:01:08

yeah, I ended up doing that, but I wondered whether there was a better way

kwladyka22:01:33

Can you recommend articles to read about security in the context of integration to third party systems? The best related with Clojure solutions. I mean there is a SaaS, which needs to fetch data from many other systems (auth for this systems is mostly e-mail + password by API and there is no read only). The question is how to secure this things from hacking. I would like to read about good articles about solutions to secure third party systems authentication data.

kwladyka22:01:18

I think the main issue is about store this data in DB

rboyd22:01:19

so database security best practices? and consider disk encryption (encrypted vols on aws, bare metal SED SSDs, ..)

kwladyka22:01:25

Hmm is there any way to outsource this responsibility? Integration with google password storage or similar magic? 😉

kwladyka22:01:03

only if third party system has it, but we are talking about systems, which have e-mail + password to auth

kwladyka22:01:29

hmm I was using vault for DevOps. Does it work without any issue for storing third party auth in Saas for users?

hiredman22:01:08

no idea, I've never used it, it just sounds kind of like what you are describing, a store designed for storing secrets

kwladyka22:01:34

yes, it can be on of choice maybe

kwladyka22:01:08

thanks, Clojure community always make me more creative 🙂 People who think by ( ) 😉

rboyd22:01:50

is that their cloud HSM?

kwladyka23:01:19

https://cloud.google.com/hsm/ I see google has something like that, but I am not familiar with that technology. It is also interesting.

kwladyka23:01:21

Thanks. I am going to sleep. Good night 🙂

👋 1