Fork me on GitHub
#clojure
<
2019-12-23
>
lvh01:12:09

Hi, I'm messing with SPARQL/RDF using aristotle. I'd like to write RDF queries that hit dbpedia/wikidata. aristotle wants an implementation of Jena's Graph interface, but all the SPARQL stuff seems to skip the Graph and just take a SPARQL query. I managed to get Jena working directly:

(let [q (QueryFactory/create "SELECT ?prop ?place WHERE {  ?prop ?place .}")
        qx (QueryExecutionFactory/sparqlService "" q)]
    (.addParam ^QueryEngineHTTP qx "timeout" "10000")
    (ResultSetFormatter/out System/out (.execSelect qx) q))
but nothing there looks like a Graph. I eventually got to a Graph by clicking around in the apidocs and came up with:
(-> "" RDFConnectionFactory/connect .fetch .getGraph)
... but I don't really know what that means and it errors out with a 400 Bad Request anyway. Has anyone tried this?

sudakatux11:12:52

Do you guys know a way to easaly copy/clone an InputStream without consuming it. OR if i really need to consume it (I think i cant scape this) how do i easaly clone it

sudakatux11:12:01

Mi idea is to clone it and then just async log its data

Ike Mawira11:12:10

Like copying it to an output stream for instance?

Ike Mawira11:12:37

Not sure if this is what you are looking for but the first example here can help: https://clojuredocs.org/clojure.java.io/input-stream#example-5542a631e4b01bb732af0a8f

sudakatux11:12:08

maybe. but. there where he is copying isnt he consuming the stream?

sudakatux11:12:27

meaning xin is consumed right?

Ike Mawira12:12:02

Yeah. Byte array is loaded to memory i guess, not quite effective for very large files

sudakatux12:12:51

cool I want it so that its not consumed. but i think i cant really do that right

Ike Mawira12:12:51

There is almost always a solution, πŸ˜‚. Lemme see what i can find incase i find something I'll share if someone won't already have.

sudakatux12:12:07

Ha thanks. Yeah i meant because I imagine that the whole idea of the stream is that its a stream so i may not know when it ends. so maybe copying without consuming it doesnt make sense Or thats what i figure

sudakatux12:12:20

i think ill just create a function that takes an input stream and returns two. consuming one and creating two out of that one

sudakatux12:12:47

makes sense?

Ike Mawira12:12:12

Yes makes sense. What i am not sure is if it will be loaded to memory when you are creating it and for every time you are creating another one out of the main one.

Ike Mawira12:12:01

If the original content is a file it would be better to use with open, return a lazy seq, do your ops on the lazy sequence before realizing them only when necessary. There is an example in the clojure docs let me try to find it.

Ike Mawira12:12:25

@U8JTE9PG8 this is the example i was talking about, if the original data is a file. Something close to https://clojuredocs.org/clojure.core/with-open#example-5ac3318fe4b045c27b7fac2f

sudakatux12:12:46

ahh interesting

Ike Mawira12:12:42

Welcome. But i guess too much of an overkill for a one liner. πŸ˜….

Ike Mawira12:12:17

Assuming, the loading to memory is done three times as stated earlier. Ps: I stand to be corrected, i haven't really worked with streams in CLJ, so i might be misinterpreting the clojure operation on Input streams, just done them in java.

sudakatux12:12:47

cool thanks.

sudakatux12:12:04

I do have a solution BTW

sudakatux12:12:33

meaning im currently consuming it loggind the data and recreating it

sudakatux12:12:59

I was just wondering if there was an easier way, or a way of avoid consuming it

sudakatux12:12:39

I dont think there is. because of it being a stream

sudakatux12:12:27

not easier really since it just a one-liner but a way to avoid consuming it

noisesmith19:12:11

I'm looking at scrollback now, but in case you didn't see this yet: FilterInputStream takes your inputstream, and returns a new inputstream that performs your extra actions on the data as the new InputStream

(import ( InputStream FilterInputStream ByteArrayInputStream))

(defn proof-of-concept
  [s]
  (let [is (ByteArrayInputStream. (.getBytes s "UTF-8"))
        filter-stream (proxy [FilterInputStream] [is]
                        (read
                          ([]
                           (println "reading, no arg")
                           (.read is))
                          ([b]
                           (println "reading, one arg")
                           (.read is b))
                          ([b off len]
                           (println "reading, three args")
                           (.read is b off len))))]
    (slurp filter-stream)))
(cmd)user=> (load-file "/tmp/streams.clj")
#'user/proof-of-concept
(cmd)user=> (proof-of-concept "hello")
reading, three args
reading, three args
"hello"

noisesmith19:12:21

for real code you likely want to implement the full interface (you can find that via javadoc), but read is enough for a focused demo

sudakatux11:12:52

Thanks im actually dealing with a FilterInputStream the body of a request

noisesmith18:12:07

sure, and you can wrap a FilterInputStream the same as you can wrap any InputStream

noisesmith18:12:20

(and somebody else aready did that before you)

borkdude15:12:25

var question 1: has anyone here ever relied on the thread-safety of alter-var-root directly?

borkdude15:12:53

var question 2: why is there a method doReset? I don't see it used anywhere

borkdude15:12:07

var question 3: what is the rev counter for?

Alex Miller (Clojure team)15:12:25

on 3, it's unused, was from some older version. there's a jira to remove that.

Alex Miller (Clojure team)15:12:28

on 2, that's an impl of the Settable interface, which Var implements

Alex Miller (Clojure team)15:12:18

I don't think the Settable stuff is used from anywhere, so that's probably legacy stuff too I'd guess

Alex Miller (Clojure team)15:12:10

on 1, not sure what you're asking

borkdude15:12:05

Well, there isn't a function in clojure called reset-var-root, only alter-var-root that can atomically swap the value of a var, so I wonder what's the reason for that: I usually only need to set the root value of a var, but not atomically using a function. Why is it important for vars to have this feature / property?

Alex Miller (Clojure team)15:12:34

all refs types have an "update" function of this shape, some also have "reset"

Alex Miller (Clojure team)15:12:51

in Programming Clojure, we (here the royal we meaning Stu, me, and via discussion Rich) call this the Unified Update Model. Has also been called the Uniform State Transition Model in the past.

Alex Miller (Clojure team)15:12:48

vars kind of fit into the model, but are a little special

borkdude15:12:32

Yes, I was aware of that. Just wondered in the case of vars where the atomically updating is used in practice by anyone. I have a Clojure interpreter which implements its own vars, but they are not atomic yet... Therefore I probably have to use locking... and if I do that, I get issues with Graal πŸ™‚

Alex Miller (Clojure team)15:12:06

ah, I understand the original question now

Alex Miller (Clojure team)15:12:36

I'd say it's unusual for many programs to rely on this directly

Alex Miller (Clojure team)15:12:30

some of the tag hierarchy / derive stuff does but that's again relatively unusual

borkdude15:12:49

good to know, thanks

Alex Miller (Clojure team)15:12:50

and is probably not typically thread-sensitive

borkdude15:12:42

Maybe I could also just use an atom as the container for the var type. Then I would get this for free

borkdude15:12:06

but in a lot of cases that's not necessary

Alex Miller (Clojure team)15:12:30

they are similar in many ways (but differ in dynamic scope support etc)

didibus16:12:18

I use alter var root mostly in tests

didibus16:12:19

I guess it probably would be rare to have multiple threads race towards doing an alter-var-root

didibus16:12:32

In practice

andy.fingerhut17:12:58

Isn't def the "reset" of Vars?

borkdude18:12:41

Yeah, but there isn't a function like swap! or reset! or alter-var-root! that does this with a var object.

didibus07:12:09

Don't think def resets the var root. I think it justs sets a new Var to the same interned symbol

borkdude08:12:52

I don’t think that’s true. E.g try inspecting the hash of the var

borkdude15:12:56

funny, it seems with-open generates one redundant expression:

(macroexpand '(with-open [a b]))
(let* [a b] (try (clojure.core/with-open []) (finally (. a clojure.core/close))))

borkdude15:12:22

Sorry, didn't expand the entire thing:

user=> (walk/macroexpand-all '(with-open [a b]))
(let* [a b] (try (do) (finally (. a clojure.core/close))))

jeko15:12:13

Hey Clojurians !

πŸ‘‹ 4
Alex Miller (Clojure team)15:12:20

doesn't that macroexpand out by the end?

borkdude15:12:43

Yes, just noticed. Sorry

metame18:12:26

any naming conventions for futures that people like besides appending -future ?

vemv19:12:09

foo-worker, foo-runner

πŸ‘ 4
metame18:12:44

e.g.

(let [names-future (future (get-names ...))
      names @names-future]
 ...)

frozenlock18:12:09

I've seen the use of * to indicate derefable values.

(let [*my-atom (atom nil)])

metame18:12:55

Yes, seen that with atoms but could make sense with all dereferables

Eduardo Mata18:12:25

Is there a library rather than http-kit that can be more reliable? Using http kit has gave me some headaches in terms of querying some API endpoints that return data but http kit treats the response as text

henrik13:12:47

Aleph is nice.

lukasz18:12:41

clj-http - it has support for automatic coercions

lukasz18:12:02

or deserialization/serialization of data

Eduardo Mata18:12:32

ahh yes. It is kind of annoying to deref my responses

fabrao21:12:44

hello all, is there any way to see a parent dependency of a lib?

fabrao21:12:31

like what this lib commons-compress-1.8.jar came from?

Alex Miller (Clojure team)21:12:40

which tool are you using?

fabrao21:12:57

I tried boot show -d

fabrao21:12:10

but it shows only 1st level

Alex Miller (Clojure team)21:12:50

sorry, don't know about boot

fabrao21:12:04

so you know in lein?

fabrao21:12:47

IΒ΄ll create project.clj to see the dep tree

fabrao21:12:16

IΒ΄m Windows guy :-

seancorfield21:12:21

@fabrao You can install clj for Windows via Scoop -- it's a nice package manager πŸ™‚

βž• 4
seancorfield22:12:36

@fabrao boot show -d shows the full dependency tree for me:

[ring "1.6.0-beta6"]
β”œβ”€β”€ [ring/ring-core "1.6.0-beta6"]
β”‚   β”œβ”€β”€ [clj-time "0.11.0"]
β”‚   β”œβ”€β”€ [commons-fileupload "1.3.2"]
β”‚   β”œβ”€β”€ [commons-io "2.5"]
β”‚   β”œβ”€β”€ [crypto-equality "1.0.0"]
β”‚   └── [crypto-random "1.2.0"]
β”œβ”€β”€ [ring/ring-devel "1.6.0-beta6"]
β”‚   β”œβ”€β”€ [clj-stacktrace "0.2.8"]
β”‚   β”œβ”€β”€ [hiccup "1.0.5"]
β”‚   └── [ns-tracker "0.3.0"]
β”‚       β”œβ”€β”€ [org.clojure/java.classpath "0.2.2"]
β”‚       └── [org.clojure/tools.namespace "0.2.10"]

seancorfield22:12:23

(that's with boot 2.7.2)

fabrao22:12:21

boot show -d
[com.climate/claypoole "1.1.4"]
[com.microsoft.sqlserver/mssql-jdbc "7.0.0.jre8"]
[com.taoensso/timbre "4.10.0"]
??? [com.taoensso/encore "2.91.0"]
?   ??? [com.taoensso/truss "1.5.0"]
?   ??? [org.clojure/tools.reader "0.10.0"]
??? [io.aviso/pretty "0.1.33"]
[lock-key "1.5.0"]
??? [base64-clj "0.1.1"]
??? [charset "1.2.1"]
[mount "0.1.16"]
[org.apache.poi/ooxml-schemas "1.4"]
??? [org.apache.xmlbeans/xmlbeans "3.0.1"]
[org.apache.poi/poi-ooxml "4.1.1"]
??? [com.github.virtuald/curvesapi "1.06"]
??? [org.apache.commons/commons-compress "1.19"]
??? [org.apache.poi/poi-ooxml-schemas "4.1.1"]
[org.apache.poi/poi "4.1.1"]
??? [commons-codec "1.13"]
??? [org.apache.commons/commons-collections4 "4.4"]
??? [org.apache.commons/commons-math3 "3.6.1"]
[org.clojure/clojure "1.10.1"]
??? [org.clojure/core.specs.alpha "0.2.44"]
??? [org.clojure/spec.alpha "0.2.176"]
[org.clojure/java.classpath "0.3.0"]
[org.jfree/jfreechart "1.5.0"]
[org.jfree/jfreesvg "3.3"]
[overtone/at-at "1.2.0"]
[seancorfield/next.jdbc "1.0.10"]
??? [org.clojure/java.data "0.1.4"]
    ??? [org.clojure/tools.logging "0.2.3"]

fabrao22:12:18

using [org.clojure/java.classpath "0.3.0"] I canΒ΄t see [org.apache.commons/commons-compress "1.18"] in dep tree

fabrao22:12:53

where is it came from?

fabrao22:12:47

dep tree of dep tree of ... ?

fabrao22:12:55

clojure use what version ? 1.19 or 1.18?

noisesmith22:12:13

the way maven deps are resolved, only one version of a given dep will be pulled in, and this run of the dep resolver picked 1.19 if the version matters, you can explicitly ask for a version before the other lib implicitly requires the lib

noisesmith22:12:07

regarding "where is it came from" - the dep is resolved recursively, the tree shows poi-ooxml as a top level dep, and the dep wasn't specified before that, so the version it asks for was used

seancorfield22:12:04

@fabrao If you believe you have a conflict, use boot show -p -- pedantic mode -- which will show you all possible conflicts, how they were resolved, and which dependency brought them in.