Fork me on GitHub
#clojure
<
2018-07-17
>
didibus07:07:49

Any way to have gen-class generate a protected method?

witek08:07:48

Hello. I need a hint for the following problem: When writing hiccup, I often have a vector containing mostly static content: [:ul [:li "A"] [:li "B"] [:li "C"]]. But sometimes I have to include values from an other sequence into my list. Here I need to include elements from the vector (def subitems ["B.1" "B.2"]) into my hiccup. Is there an elegant way to place some code between [:li "B"] and [:li "C"] which inserts elements from subitems wrapped in [:li ...]s?

witek08:07:49

I am looking for something which would be used like that: [:ul [:li "A"] [:li "B"] (magic-for [subitem subitems] [:li subitem]) [:li "C"]].

chrisblom08:07:57

It’s not really idiomatic but you can use unquote-splice for this:

chrisblom08:07:02

(let [subitems  ["B.1" "B.2"]]
  `[:ul [:li "A"] [:li "B"] ~@(for [subitem subitems] [:li subitem]) [:li "C"]])

🙏 4
mitranim09:07:56

@witek Hiccup will automatically unwrap non-vector sequences, splicing them into the enclosing element

mitranim09:07:19

So I'm not sure why you wouldn't just use a (for ...)

henrik09:07:59

I have big, icky XML blobs to extract data from and structure into sensible EDN. Are zippers the way to go?

reborg10:07:47

@U06B8J0AJ, is the goal of the output EDN to contain all of the XML content with a different shape? Or is the edn going to be more of a flat structure?

henrik10:07:51

The EDN is going to be heavily flattened, shrunk, and restructured.

henrik10:07:11

Large chunks ignored or thrown away.

reborg10:07:38

I see, you could give a go at [clojure.xml :refer [parse]] to parse it, and then use xml-seq to pick the bit you are interested in. A few examples here https://clojuredocs.org/clojure.core/xml-seq

reborg10:07:42

With zippers or postwalk, I guess you'd probably have to write more complicated code.

roklenarcic10:07:58

The first question is: can you fit that DOM into RAM. I've seen 2 MB xmls expand to over 100 MB DOM trees after java's xml parser was done.

henrik10:07:16

So far, I’ve parsed the XML into an intermediate format. For example;

<journal-meta>
<journal-id journal-id-type="publisher-id">IBM</journal-id>
<journal-id journal-id-type="hwp">spibm</journal-id>
<journal-title-group><journal-title>International Bulletin of Mission Research</journal-title></journal-title-group>
<issn pub-type="ppub">2396-9393</issn>
<issn pub-type="epub">2396-9407</issn>
<publisher>
<publisher-name>SAGE Publications</publisher-name>
<publisher-loc>Sage UK: London, England</publisher-loc>
</publisher>
</journal-meta>
Gets turned into
[[:journal-meta
  [[:journal-id "IBM" [[:journal-id-type "publisher-id"]]]
   [:journal-id "spibm" [[:journal-id-type "hwp"]]]
   [:journal-title-group
    [:journal-title "International Bulletin of Mission Research"]]
   [:issn "2396-9393" [[:pub-type "ppub"]]]
   [:issn "2396-9407" [[:pub-type "epub"]]]
   [:publisher
    [[:publisher-name "SAGE Publications"]
     [:publisher-loc "Sage UK: London, England"]]]]]]

henrik10:07:50

Given this, I wrote a function called cherry-pick, which takes a set of keywords, traverses the entire structure, and returns any hits for those keywords. For example, running (cherry-pick #{:issn}) Returns

[[:issn "2396-9393" [[:pub-type "ppub"]]]
 [:issn "2396-9407" [[:pub-type "epub"]]]]

henrik11:07:14

So the general strategy is, flatten the XML a bit, and have a way of tearing out pieces of interest from it. Given the cherry-picked items, do some more context sensitive conversions. But I’m not at all certain that this is a good way to go about it.

reborg11:07:19

A comparison with xml-seq

(->> xml 
 .getBytes 
 io/input-stream 
 xml/parse 
 xml-seq 
 (filter (comp #(= :issn %) :tag)))

;;({:tag :issn, :attrs {:pub-type "ppub"}, :content ["2396-9393"]} 
;; {:tag :issn, :attrs {:pub-type "epub"}, :content ["2396-9407"]})

reborg11:07:38

And yes, you should pay attention at the size of the input XML. If you get OOMs you could go data.xml lib (which is lazy)

👍 4
henrik11:07:59

Given that I have to search the entire structure one way or another, won’t the lazy version become reified anyway?

reborg11:07:03

well if you pipe the xml through one side and get the processed version on the other, the full XML doesn't need to stay in memory in full

henrik11:07:28

So, filter, in the example above, won’t cause OOM?

reborg11:07:46

only if the parsing is done with data.xml instead of the core xml namespace

henrik11:07:32

Gotcha, thank you

reborg11:07:21

np, you might need to tweak the filter if you use data.xml, but the process is more or less the same. It looks to me you can avoid the intermediate step.

henrik11:07:41

Perhaps, but I still need a version of filter that can act on nested data structures, right?

henrik11:07:22

Or does xml-seq flatten the data structure?

reborg11:07:52

xml-seq is what is flattening the structure.

henrik11:07:24

Alright, that’s pretty cool. I’m starting to understand now.

henrik11:07:40

Yeah, it looks like I can throw out the intermediate step.

henrik11:07:50

Ah yes, xml-seq definitely wins in terms of ergonomics. Thank you.

👍 4
ghadi12:07:35

I'd use xpath, tbh

ghadi12:07:07

Depending on if your blobs fit in memory

henrik12:07:31

The one’s I’m looking at right now do, but there’’s no guarantee about future ones.

henrik14:07:25

@U050ECB92 What lib do you use for xpath?

ghadi14:07:21

raw interop

ghadi14:07:26

I can share you some code

ghadi14:07:00

basically some helpers to execute an XPath and transform the stuff back to maps and vectors. You might want to try clj-xpath, looks decent

henrik14:07:47

I’d love an interop example, and I’ll have a look at clj-xpath as well.

Jeremy12:07:35

Hi folks, can any tolitius/mount users help me out? I'm trying to dispatch a multimethod using (constantly name-of-defstate), where name-of-defstate is just a keyword. Is this possible? I can't see any reason the dispatch function should be called before I do (mount/start), but it's not working.

oscar17:07:47

Did you mean symbol instead of keyword? Also, if your dispatch is that, then it will always dispatch to the same value. constantly returns a function that always returns its argument (it doesn't call it). You probably want something like (defmulti my-fn <name-of-defstate>).

Jeremy20:07:50

Hi oscar, I meant name-of-defstate returns a keyword from its :start, e.g. :dev to indicate dev mode globally in my application. I had hoped (defmulti my-fn (constantly mode-defstate)) would dispatch to the (defmethod my-fn :dev ...).

oscar20:07:58

I see. You probably want (defmulti my-fn (fn [& _] mode-defstate)) then.

oscar20:07:49

constantly is called before (mount/start) so it's getting passed the wrong value as its parameter.

oscar20:07:33

Putting mode-defstate in a fn ensures that it doesn't get resolved until call time.

Jeremy00:07:35

thank you; I guess I assumed that constantly was doing the same thing; obviously not

roklenarcic13:07:32

How can I get ns metadata?

athos13:07:33

something like (meta (find-ns 'clojure.core))?

roklenarcic13:07:47

that works, thanks

idiomancy15:07:06

can someone check my sanity? A.B.C(D) in clojure interop is (.. A B (C D)) or (.. A B C D)?

idiomancy15:07:33

or... is it ((.. A B C) D)

idiomancy15:07:59

okay, thanks 😂

theeternalpulse17:07:26

I know this is probably a sensitive subject, but is there a way to colocate clojure tests with the files they test. I know the src/test separation is more than just superficial, but I've always found putting tests in a special "far away" directory to never as be as helpful as keeping them next to their test on files.

hiredman18:07:52

clojure.test doesn't make a distinction between test namespaces and code namespaces, you can put clojure.test tests anywhere

hiredman18:07:09

but you will end up shipping test code in your final artifact if you mix them

huthayfa19:07:33

hello, how do I get the name of the function being excuted as string

Alex Miller (Clojure team)19:07:21

there actually is no canonical way to do that

Alex Miller (Clojure team)19:07:57

functions are compiled into classes and there is a demunge function that can be used to turn that class name into a string

Alex Miller (Clojure team)19:07:14

however, the function name => class name transformation is potentially lossy so in some cases you will not get the identical function name you started with

huthayfa19:07:16

I think the function meta provide some info like the function name but I can't remember

Alex Miller (Clojure team)19:07:57

you may have a :name meta for the function var

Alex Miller (Clojure team)19:07:01

(-> #'map meta :name str) ;; => "map"

huthayfa19:07:31

yes this works but I still have to pass the function var

lilactown22:07:56

I'm struggling to get it to work. it seems like the new relic agent is not picking up my newrelic.yml file

lilactown22:07:23

okay, maybe it has to do with trying to use the jar from maven

lilactown22:07:41

new question: how do I load a local jar in my lein project so that it's available when I start a REPL?

jcr22:07:16

lein install?

noisesmith22:07:22

add the repo that the jar is in to your maven repos, or worst case install it to your local maven cache with lein install

noisesmith22:07:43

lein install is iffy because it causes unpredictable dep resolution