Fork me on GitHub
#clojure
<
2017-12-28
>
haus00:12:01

@radomski are you already using reader conditionals?

Ryan Radomski00:12:44

Yeah I tested out my reader conditionals in a chestnut project and they worked in both runtimes.

Ryan Radomski00:12:21

I just never actually packaged a pure cljc project. Does the default lein template work fine for both runtimes?

haus00:12:39

that part i’m not sure about

wiseman00:12:18

https://github.com/stuartsierra/component looks like a minimal example of a (not quite pure) .cljc library

Ryan Radomski00:12:03

I should have looked at Sierra's components. It never clicked that I had them on the back and the front with the same namespace. It looks like this library has a project.clj a lot like mine as well. Thanks much!

anisoptera01:12:38

hey, has anyone successfully used a go block in clojure >=1.7.0? i am having trouble with it not closing over its scope properly, so (let [c (chan)] (go (prn (<! c)))) gets me Unable to resolve var: c

anisoptera01:12:55

this is right out of the tutorial, and it works in clojure 1.6

anisoptera02:12:05

but that seems like an awful long time for something so fundamental to be broken

anisoptera02:12:12

so i’m open to it being totally my problem

noisesmith02:12:55

the weird thing about that block of code is that core.async should treat it as a noop returning a channel that never delivers

noisesmith02:12:02

since nothing can possibly use c

anisoptera02:12:34

i mean, there’s a more complicated version of that code, which also fails with the same issue, which actually does do a real hello world with the channels, in the tutorial

anisoptera02:12:44

but i stripped it down to just that being enough for it to fail with the same issue

noisesmith02:12:11

Clojure 1.9.0
+user=> (require '[clojure.core.async :as > :refer [go chan <!]])
nil
+user=> (let [c (chan)] (go (prn (<! c))))
#object[clojure.core.async.impl.channels.ManyToManyChannel 0x1e5e2e06 "clojure.core.async.impl.channels.ManyToManyChannel@1e5e2e06"]

anisoptera02:12:39

i accidentally had written a lot of actually working code before realizing this issue, because i had defed a channel with the same name as i was using inside go blocks

anisoptera02:12:46

ok well what the crap

noisesmith02:12:57

what core.async version are you using?

anisoptera02:12:27

i tried a lot of releases

noisesmith02:12:37

that's the version I was using

noisesmith02:12:34

this is straight up clojure + core.async running from a custom uberjar (project here https://github.com/noisesmith/bench - I just run lein uberjar then make the jar an executable)

anisoptera02:12:52

option-bot.core> (require '[clojure.core.async :as > :refer [go chan <!]])
nil
option-bot.core> (let [c (chan)] (go (prn (<! c))))
ExceptionInfo Could not resolve var: c  clojure.core/ex-info (core.clj:4739)

noisesmith02:12:32

are you using leiningen? if so try using lein clean then starting a new repl

noisesmith02:12:53

sometimes things can get wonky with core.async and lein caches when versions of things change, in my experience

anisoptera02:12:04

same deal 😞

noisesmith02:12:50

are you using any weird plugins? is there a chance that lein deps :tree shows a version conflict that would give you the wrong version of core.async?

anisoptera02:12:20

nah, it’s pretty simple

anisoptera02:12:23

[clojure-complete "0.2.4" :exclusions [[org.clojure/clojure]]]
 [org.clojure/clojure "1.9.0"]
   [org.clojure/core.specs.alpha "0.1.24"]
   [org.clojure/spec.alpha "0.1.143"]
 [org.clojure/core.async "0.3.465"]
   [org.clojure/tools.analyzer.jvm "0.7.0"]
     [org.clojure/core.memoize "0.5.9"]
       [org.clojure/core.cache "0.6.5"]
         [org.clojure/data.priority-map "0.0.7"]
     [org.clojure/tools.analyzer "0.6.9"]
     [org.clojure/tools.reader "1.0.0-beta4"]
     [org.ow2.asm/asm-all "4.2"]
 [org.clojure/tools.nrepl "0.2.12" :exclusions [[org.clojure/clojure]]]
 [self/ib "9.73.01-SNAPSHOT"]

anisoptera02:12:46

i mean, there is one custom java api library, but that shouldn’t do anything like this, i wouldn’t think

noisesmith02:12:54

I'm going to see if a project with those deps gives me the same issue

noisesmith02:12:11

(ignoring self/lib that is)

anisoptera02:12:44

i just did lein repl and that works

anisoptera02:12:47

so it’s just cider that’s the problem

noisesmith02:12:11

cider is clearly doing something goofy

noisesmith02:12:28

thanks for figuring it out before I had to do more work heh

noisesmith02:12:02

so I guess the answer to "are you using any weird plugins" is "we didn't realize how weird cider is"

anisoptera02:12:02

guess i’ll try updating that

noisesmith02:12:31

Tooling, the cause of, and solution to, every problem we have when programming clojure.

noisesmith02:12:52

(to paraphrase homer simpson)

anisoptera02:12:04

and i thought every issue i had with it had been fixed now that cider could pass its plugin versions into lein

noisesmith02:12:33

I stopped trying to figure that magic out a long time ago ¯\(ツ)

anisoptera02:12:37

updated cider, it all appears to work

anisoptera02:12:41

lost my awesome scratch buffer

anisoptera02:12:02

thanks for being a sanity check

anisoptera02:12:16

i was like “there’s no way this has actually been broken for the last three years, right?”

anisoptera02:12:11

so here’s a more general question

anisoptera02:12:40

what’s a good way to represent a record that requires multiple api calls (and callbacks) to be fully realized?

anisoptera02:12:48

like, i have a representation of a stock

anisoptera02:12:11

and there’s a bunch of static info about it that’s just intrinsic to it

anisoptera02:12:42

but then also it kind of has a price, right? but to get the price, that’s the result of some other call

anisoptera02:12:07

so i have been putting that behind a future

noisesmith02:12:16

a good functional technique is to put the price and the time in there together, and make sure all price queries are explicit about time windows they accept

noisesmith02:12:51

and sure, you can put a future in, you could also have a function that returns a future that returns a new instance of your record (with a newer time stamp) when realized

noisesmith02:12:59

that ensures the parts of the record are all in sync

anisoptera02:12:35

oh and then i’d have some fn get-price (or maybe use conform) and it’d get me the price that was cached if it was fresh enough

anisoptera02:12:39

or go get a new one

noisesmith02:12:44

also, you could use a TTL cache (clojure core.cache provides this) and retrieve the price from the cache

noisesmith02:12:55

if the record is created recently enough, it's returned, otherwise queried

noisesmith02:12:56

the nice thing with core.cache is that it doesn't force a specific storage - it just takes a hash-map and returns a new one, and you can put this in a local that propagates via recur, or put it in a ref, or an atom, or even a proper db if you serialize it right - it should just work

noisesmith02:12:13

(as long as you always use the new hash-map it returns of course)

anisoptera02:12:30

that’s super handy

anisoptera02:12:47

because yeah i’m making a lot of wasteful API calls

anisoptera02:12:08

i basically just hacked together some bullshit that worked for now with promises/futures

qqq02:12:11

I'm trying to read 10k bytes from a file, and I don't wnat to loop; I want to make on3 call and have java read it for me

noisesmith02:12:39

if you know it's always 10k bytes or less why not pass a 10k byte-array to the read method?

noisesmith02:12:20

it's saying "look at the method foo on class bar" - it's just phrasing it kind of weird

noisesmith02:12:13

I use core.cache in front of mongo documents encoded via transit

anisoptera02:12:24

now i’m trying to go back and use core.async channels to model the continuous updates

noisesmith02:12:34

the core.cache protocols do all the cache ttl logic, and transit handles the serialization, and mongo the storage

noisesmith02:12:25

regarding mongo - I wouldn't necessarily recommend for a new project, it was inherited as part of the team expertise if that makes any sense

anisoptera02:12:40

yeah i haven’t even begun to think about a persistence layer

anisoptera02:12:50

i’m doing like super elementary algorithmic trading

anisoptera02:12:03

mostly because i want to stop having to wake up at like 4am to catch market opens

anisoptera02:12:36

so far clojure’s concurrency stuff has been as good as i expected for this

noisesmith02:12:38

yeah, core.cache also does simple in-memory cache with ttl which is what I'm sure you actually want - for those mongo caches I'm measuring the ttl in days not minutes

anisoptera02:12:48

it’d be like seconds

anisoptera02:12:10

but there are some other queries that i might want to store for longer

anisoptera02:12:26

like, price data, seconds, but the actual existence of the contracts i’m interested in

anisoptera02:12:36

that can be cached for a long time

noisesmith02:12:36

also the simplest solution would be to use an atom for your cache, but if you start to hit contention slowdowns you could use multiple refs instead of an atom with multiple keys in it

anisoptera02:12:53

i am also thinking like

anisoptera02:12:10

oh, instead of handing out refs like candy and then trying to update them in place

anisoptera02:12:39

i guess i should make a function that is like get-contract and then have that access the cache or synthesize a new contract info object

noisesmith02:12:58

using lots of refs does work (and performs better under heavy write load) it's just a more complex design than a hash-map in an atom with a key for each "thing"

anisoptera02:12:01

i have been thinking it would be kind of neat to have an agent for every contract i’m keeping track of

anisoptera02:12:22

and then just deref the contract for the current state

anisoptera02:12:39

but maybe using an explicit cache would be much better

anisoptera02:12:02

oh, that’s what you were saying, though

anisoptera02:12:07

a ref for every cache entry?

noisesmith02:12:34

caching logic is notoriously tricky to get right, and core.cache helps a lot with that; I was surprised - I did a big cache with a lot of features built on core.cache and it worked without a hitch after the trivial data bugs were fixed

noisesmith02:12:09

@anisoptera I'm trying to work out what this would require - it's a lot simpler to add keys on demand to an atom, but you could also hypothetically manage multiple refs but then you have the question of what tracks these on-demand refs and then I'm like "refs inside an atom" and no that is insanity

anisoptera02:12:26

that’s where i went too

anisoptera02:12:29

it was like aaaaa

anisoptera02:12:45

because the idea of a record that magically updates itself

anisoptera02:12:19

i guess for right now, it’s not a big deal, i can just have a single atom with a map

anisoptera03:12:05

wow, that caching lib is great, just implementing a really naive caching layer in my calls was enough to massively reduce the number of API roundtrips

qqq08:12:29

is there a way of writing call (.readInt ...) on this data-input-stream 10 times that is better than loop-recur ? (note that .readInt changes the 'pointer' in the file buffer, so we need to be careful with regards to laziness

Bravi11:12:23

Hi everyone. I’m using a chestnut template and I have this simple route

(defn home-routes [endpoint]
  (routes
    (POST "/api/get-name" _ get-name)
    (resources "/")
    (not-found "<h1>Page hello</h1>")))
whenever I try to post to this route from FE (cljs / re-frame), it sends OPTIONS type request first and it fails straight away with 404 and then the POST is not being sent afterwards. When I manually add
(OPTIONS "/api/get-name" _ get-name)
  (POST "/api/get-name" _ get-name)
it returns with 200 but then POST is not being sent afterwards

burke12:12:25

Your browser is sending OPTION request to make sure the planned POST request is acceptable. This may help you understand option request: https://developer.mozilla.org/de/docs/Web/HTTP/Methods/OPTIONS Check the network/console log of your browser to see which headers/methods/origins in the options request are asked for and then make sure your OPTIONS endpoint in your backend sends a valid option response. I think for most cases there are ring-middlewares to do this

jmckitrick15:12:50

Hi all. I have a spec question. I think I know the answer, but I want to confirm. If I'm spec'ing a map, and the keyword is :fooBar, then the spec must have the same name, correct? Actually, it would be namespaces, so ::fooBar, correct? I need to know if I can remap a keyword to a spec with a different name, or if that defeats the purpose of readability.

noisesmith15:12:11

you can spec un-namespaced keys with :req-un and :opt-un https://clojure.org/guides/spec

noisesmith15:12:33

but there are good reasons to use namespaced keys

jmckitrick15:12:55

Well, it’s not actually the namespacing. I’m fine with that part. I’m asking if I could have a ::foo-bar spec for the :fooBar key

jmckitrick16:12:12

Or, suppose you have 2 different API returns in the same domain, but that have the same keyword. Such as :results but where the spec for each of those is different.

taylor17:12:55

the names must match for the keys spec to work

daveliepmann17:12:10

I'm looking for a few more folks to share their workflow in detail over at ClojureVerse: https://clojureverse.org/t/share-the-nitty-gritty-details-of-your-clojure-workflow/1208 Lots of great productivity hacks both big and small. Both Clojure beginners and experienced devs seem to be getting a lot out of it. clj

jmckitrick17:12:03

@taylor thanks for that answer. What would be the best way to spec data where the same keyword is used in different ways? For example, if I’m searching for books and music, both might return data with ‘results’ in it. But the shape of those results will be different.

jmckitrick17:12:58

(s/def ::book-results (s/keys :req-un [::count ::results]))

jmckitrick17:12:21

(s/def ::music-results (s/keys :req-un [::count ::results]))

jmckitrick17:12:55

So ::results actually need to point to 2 different specs, but the keywords still need to match the data they are based on.

jmckitrick17:12:11

Unless I can do this:

jmckitrick17:12:41

(s/def ::book-results (s/keys :req-un [::count ::book/results]))

jmckitrick17:12:04

I’ll try it…

taylor18:12:24

@jmckitrick is there anything else in that map that indicates whether it’s a book or music? if so, maybe you could use multi-spec

jmckitrick18:12:24

I think I figured it out @taylor.

jmckitrick18:12:28

(s/def :my/result int?)
(s/def :your/result pos-int?)
(s/def ::test-spec-1 (s/keys :req-un [:my/result]))
(s/def ::test-spec-2 (s/keys :req-un [:your/result]))

jmckitrick18:12:15

(s/valid? ::test-spec-1 {:result -2})

jmckitrick18:12:40

(s/valid? ::test-spec-2 {:result -2})

jmckitrick20:12:50

Does spec have any impact on parsing XML?

jmckitrick20:12:07

Well, I’m about to pitch my CTO on Clojure and spec, and we consume a lot of XML API’s. I’d like to know definitively how they relate, and if spec is an advantage here or not.

dominicm20:12:27

@jmckitrick I don't think there's much advantage, you could spec those XML APIs, and generate clojure.data.xml data structures automatically from them, to ensure that you have high coverage of the various data shapes that could be thrown at you from those APIs.

dominicm20:12:02

Spec is really "one layer down" from application APIs, and is more centered around function APIs.

jmckitrick20:12:33

Hmm. I see spec as a great way to validate calls to third-party API’s.

jmckitrick20:12:50

I’m trying to talk a Scala CTO into allowing some Clojure pilot projects. A big part of the argument is what spec brings to the table.

seancorfield20:12:35

When you say "consume a lot of XML APIs" do you mean you POST XML to them or you get XML responses back? In either case you could convert from/to a Clojure data structure and spec validate that Clojure data structure.

jmckitrick20:12:39

And we make quite a few XML calls.

seancorfield20:12:30

One of the reasons we initially tried Scala was that we had a problem that involved reading a SQL DB and generating XML to POST to a search engine -- and Scala has XML literals native in the language.

jmckitrick20:12:39

It will be a mix of both. I want to demo the worst-case scenario, even if we end up working with JSON.

seancorfield20:12:17

When we switched to Clojure, we used Hiccup to generate XML and that just transforms Clojure data structures so those could, in theory, be spec'd.

jmckitrick20:12:59

Nice. I think we want to demo how we can consume changing API’s in Clojure more easily than in Scala.

jmckitrick20:12:27

So it’s more a concern with what we get from the vendor, than what we send them.

seancorfield20:12:05

Converting XML to Clojure data structures is pretty painful in my experience.

jmckitrick20:12:05

I’m working on a live coding demo with spec and JSON API returns.

jmckitrick20:12:20

XML is a real-world concern.

jmckitrick20:12:34

Cause some vendors still use it, and won’t be changing anytime soon.

dominicm20:12:37

@seancorfield Oh really? We've been using clojure.data.xml internally for docbook stuff & it's been fairly good.

seancorfield20:12:30

We use clojure.data.xml in one place for a simple XML API response and also with clojure.data.zip.xml in another place for parsing a complex XML document -- lots of manual code mapping from the XML structure to the Clojure data structure we want.

jmckitrick20:12:08

Where is XSLT when you need it? 😉

dominicm20:12:14

I didn't use it for xml, but https://github.com/halgari/odin/ had some nice helpers for xml, and might be a suitable XSLT-like tool.

noisesmith20:12:42

recently I made a toy clojure program that plays the wikipedia philosophy game (as a demo of a debugging library I wrote) and tree-seq plus clojure.xml was good enough for the little dumb thing I was doing https://github.com/noisesmith/philoseek

seancorfield20:12:54

We have lots of stuff like (->boolean (or (zx/xml1-> node :required zx/text) false)) and (->long (or (zx/attr node :minimum) 0)) and what is basically a custom recursive descent parser for the XML format 😞

seancorfield20:12:28

(since XML is all plain text and we want Boolean, Long, Date etc from it)

jgh20:12:54

lol why does yours have such a huge picture compared to everyone else

flowthing20:12:10

That sucks so bad. 😛

flowthing20:12:27

I can’t remove the attachment on mobile, either!

noisesmith20:12:43

brb setting my gh profile pic to longcat

jgh20:12:48

ah just put it up we’ll get over it

flowthing20:12:04

Whatever, can’t be bothered. I’d rather not plaster my ugly mug here again. 😛

dominicm20:12:10

Better? 😄

flowthing20:12:14

Thanks! Yes.

dominicm20:12:18

fwiw, I didn't see it as big (but I tweaked stuff a little)

jgh20:12:28

well when you posted it initially it didnt have such a big picture, it’s just when flowthing posts it lol

jgh20:12:48

shows up normally for me too (tested in my dm)

flowthing20:12:51

Though in the spirit of full disclosure, I wouldn’t call that library exactly “battle-tested”… I just wanted to see whether you can write XSLT with parentheses instead of angle brackets, more or less. 😛

seancorfield20:12:53

I could always turn off previews from http://github.com but the summary is often useful (even if the picture not so much)

flowthing20:12:41

But the XPath bits might be useful for querying XML documents, I guess.

flowthing20:12:35

@seancorfield yeah, no need, it’s just weird that Slack uses that humongous version of my profile picture every time… gotta try to figure out why sometime.