Fork me on GitHub
#clojure
<
2018-07-11
>
ghadi00:07:09

https://gist.github.com/ghadishayban/0ac41e81d4df02ff176c22d16ee8b972 A helper called reify-SAM that helps Clojure code participate in "functional interfaces" AKA the Java 8 lambda interfaces AKA "Single abstract method" classes

(import 'java.util.function.Predicate)
(reify-SAM Predicate
  [v]
  (even? v))
It allows you to omit the name of the single method you're implementing, as Java does.

👍 8
roklenarcic08:07:06

How would I dynamically load a namespace and return the list of all the functions declared in it?

mpenet08:07:08

ns-publics comes to mind

roklenarcic09:07:43

I need to make sure ns is loaded first though

noisesmith15:07:36

perhaps (doto ns-sym require ns-publics)

noisesmith15:07:03

also ns-publics returns a hash-map of all publics, not just functions

noisesmith15:07:38

and in practice, most consumers want something more like the keys (symbols) or vals (vars) of the ns-publics map

cddr12:07:39

Can you specify a dependency on a private project using deps.edn?

Alex Miller (Clojure team)13:07:26

yes, depending what you mean by “private project”

cddr14:07:19

One that publishes to e.g. artifactory

cddr14:07:42

A jar that is

cddr14:07:59

Ah perfect. Thanks

joefromct15:07:31

Hi, is there a way to make zipmap force creation of APersistentArrayMap (`array-map`) so that order is retained? It looks like depending on size sometimes it makes a hashmap instead.

ghadi15:07:21

you shouldn't rely on insertion order of maps

joefromct15:07:48

ok, i guess i shouldn't be using maps then

ghadi15:07:23

Dunno. What's the surrounding use-case?

joefromct15:07:41

i need to output a file based on a spec, so, for instance column 1 needs to be where column1 needs to be...

joefromct15:07:49

there is an intermediate map for some transformations

joefromct15:07:05

i guess i could store the ordinal and then at the end re-order each row

joefromct15:07:09

but that seems like overhead

andy.fingerhut15:07:14

There are alternate map implementations that can remember the order that insertions occurred, but there is no way to make a function like zipmap return such a map.

joefromct15:07:01

but array-map should retain order though it seems

joefromct15:07:09

If i just build my own

ghadi15:07:25

i still wouldn't rely on that. don't worry about overhead, make it correct first. Is this like a csv that is transformed and then re-output?

ghadi15:07:27

As the last step after transformation, you can reorder the maps (which represent rows, associatively) by doing this:

andy.fingerhut15:07:08

Yeah, array-map is 'fragile' in the sense that you can create one, but if you then assoc one more key into it, the return value could be a hash map that no longer has the ordering of the input map.

joefromct15:07:26

yeah i won't touch it, although i might update-in

ghadi15:07:27

(let [column-order [:col1 :col2 :col3 :name :whatever]]
  (map row column-order))  ;;;;; row is a hashmap

joefromct15:07:38

(which i guess would create a new potential hash map 😕 )

ghadi15:07:49

that's the single row case ^

joefromct15:07:55

ok, i'll do it ^ way that seems like the best scenario

ghadi15:07:31

(the terminology is confusing because we're talking about hash maps and the clojure.core/map function)

joefromct15:07:41

so i'll still have a problem, so,

joefromct15:07:18

the map in question, i'm slurping from an edn file and it's called :fields .... with a key/value for field type.

joefromct15:07:41

when i read-string on that fields map it's a hashmap, so poof my oder is gone

joefromct15:07:54

I guess i'll try to store it in the EDN as a vector of pairs

dpsutton15:07:24

i think @ghadi was saying just read the data into a map and when you emit it have a vector that dictates the order of the fields. the map never has order. your emitter has an order given to it to dictate in which order to ask the map for values

andy.fingerhut15:07:18

Well, some types of map data structures preserve order of one kind or another (e.g. sorted-map, ordering-map), but the EDN reader cannot be made to use those without creating your own modified versions of those readers, AFAIK.

andy.fingerhut15:07:45

so in practice, unless you want to do those kinds of surgery, a data structure that is sequential is better to use than a map when order is important to you.

leblowl17:07:52

So in clojure.spec docs, it talks about how spec maps are only concerned with membership (https://clojure.org/about/spec#_sets_maps_are_about_membership_that_s_it). Then how can we handle cases where the values of certain keys depend on the values of other keys? I am validating input on the edge of my app and I am having a difficult time putting my requirements into spec logic. On updates I receive an id and name key. The name needs to either not exist yet (be unique) or match the current name associated with the id in the database. Any ideas on how I could implement this only in spec? Is multi-key validation outside of the scope of spec? Obviously you could attach a predicate to the map itself, but according to the docs I shouldn't be dealing with the values of keys in a map spec. Thanks

ghadi17:07:48

as you say, you can do cross-key validation by attaching predicates s/and to the map. In that case it's not violating the tenet around "maps are sets of keys".

ghadi17:07:07

"not exist yet in the database" is not something you should express with spec

👍 4
leblowl17:07:33

@ghadi thanks. ok, so pretty much any validation that depends on existing state (e.g. in a database) should be done outside of spec as a separate step.

ghadi17:07:50

yeah E_OUT_OF_SCOPE

markw17:07:02

how would one destructure this? `(def foo {[0 0] {:size 85, :used 64, :avail 21, :used-pct 75}})` ` (let [{[x y] {:keys [size used avail used-pct]}} foo] [x y size])`

markw17:07:49

sorry...

(def foo 
 {[0 0] {:size 85, :used 64, :avail 21, :used-pct 75}})
(let [{[x y] {:keys [size used avail used-pct]}} foo]
  [x y size])

👍 4
noisesmith17:07:51

you can't destructure a key in a map like that

noisesmith17:07:07

if there were two keys in the map, how would it know which one to destructure?

markw17:07:09

ok i feel better then

noisesmith17:07:36

if you call seq (and are certain there isn't a second k/v) you could destructure that though

markw17:07:58

i was trying to avoid doing first/rest and destructuring those, just for the sake of brevity

markw17:07:05

but it's probably more clear in this instance

noisesmith17:07:17

I'm not talking about using first/rest

markw17:07:24

i know what you're saying

markw17:07:30

convert to seq destructure that

markw17:07:12

something like this:

(def foo 
 {[0 0] {:size 85, :used 64, :avail 21, :used-pct 75}})
(let [[[[x y] {:keys [size used avail used-pct]}]] (seq foo)]
  [x y size])

ghadi17:07:52

In general, I avoid destructuring more than one level for readability

markw17:07:06

yeah that's a mess I agree

noisesmith17:07:19

it might be more meaningful to have something like {:index [0 0] :value {...}}

noisesmith17:07:34

conveying what that key/value pair actually intends to describe

markw17:07:05

that definitely simplifies destructuring, but makes getting the values via coordinates a bit goofier

noisesmith17:07:20

right, I don't know your domain

markw17:07:33

i think maybe i'll just stop going overboard on the destructring when it doesn't fit well

markw17:07:53

i agree w/ ghadi, whenever i try to nest beyond one level... it loses readability

noisesmith17:07:04

it's not like multiple destructure calls are more expensive than a single nested one on one line

markw17:07:19

fair point

noisesmith17:07:29

(well not in a way that's likely to be significant at least?)

noisesmith17:07:46

be sure you aren't gaming the "lines of code" metric by making one line too complex

markw17:07:46

i like to use it when it's reasonably readable mainly for the information it gives me in the function signature

petr.mensik20:07:23

I am calling some Java code via interop

(defn- add-image-to-page
  [doc pdf-image]
  (let [image (PDImageXObject/createFromByteArray doc pdf-image nil)]
  ; code omitted for clarity
  (with-open [stream (PDPageContentStream. doc page)]
      (.drawImage stream image x y scaled-width scaled-height))))
  
; Calling function here
  (with-open [doc (PDDocument.)]
    (run! #(add-image-to-page doc %) images)
This one works fine, however if I move line with creating image into the function passed to run! it blows with IllegalArgumentException on .drawImage
(defn- add-image-to-page
  "Adds image as a page to the document object"
  [doc image]
  ; Code omitted
  (with-open [stream (PDPageContentStream. doc page)]
    (.drawImage stream image x y scaled-width scaled-height))))
  
  ; Calling function here
  (with-open [doc (PDDocument.)]
    (run! #(add-image-to-page doc (PDImageXObject/createFromByteArray doc % nil)) images)
I am really getting desperate on this one. I tried to print all arguments passed into the drawImage and they are the same for both functions, also the code is same except creating image line. so what could be wrong here?

hiredman20:07:10

what is images?

petr.mensik20:07:24

it's a vector of byte arrays

petr.mensik20:07:41

Like I said, everything is the same except the line (PDImageXObject/createFromByteArray doc pdf-image nil) moved from add-image-to-page directly to the run!

hiredman20:07:28

(set! *warn-on-reflection* true)

hiredman20:07:52

what is happening is because you moved image out of the function, you are losing type inference on it, so the compiler is turning the call to drawImage in to a reflective call, and the reflective call is failing for whatever reason (clojure.lang.Reflect is picking the wrong method likely)

hiredman20:07:20

warning on reflection will show you the difference, type hinting image should fix it

petr.mensik20:07:28

@hiredman nice, that really helped 🙂

petr.mensik20:07:48

thanks, didin't know about this issue

denik21:07:47

I’m writing an ebook parser in clojure. Once unzipped ebooks are plain HTML. However, the problem is that the HTML is mostly flat, like this (in enlive)

(def ebook
  [{:class "indent" :content "ignore"}
   {:class "indent" :content "ignore"}
   {:class "START HERE" :content "Key Lessons"}
   {:class "indent" :content "ignore"}
   {:class "indent1" :content "ignore"}
   {:class "bl" :content "key lesson 1"}
   {:class "bl" :content "key lesson 2"}
   {:class "indent1" :content "ignore"}])
I’m want to structure that data so that a header contains the children writing a range and filter query like:
(query ebook {:from   #(= "START HERE" (:class %))
              :filter #(= "bl" (:class %))
              :to     some?})
that should return
=> {:class    "START HERE"
    :content  "Key Lessons"
    :children [{:class "indent" :content "key lesson 1"}
               {:class "indent" :content "key lesson 2"}
               {:class "indent1" :content "key lesson 3"}]
    }
Now I feel like I’ve implemented something like this a million times. Is there a library (even non-clojure) and query syntax to take unstructured data and give it structure. I’m aware I could use spec but spec likes to check everything and the inconsistencies in ebook make it hard to spec all of it out. So I’m looking for something that just finds what matches and goes from there.

jsa-aerial14:07:30

Take a look at #specter

denik14:07:01

thanks jsa, I’ll post it there.

hiredman22:07:35

actually spec might work well for that

denik22:07:49

theoretically yes, but the problem is specs objective is spec things, in other words, to identify what they are. I’m looking for something that is more like a search, a query, that infers relationships through sequence.

hiredman22:07:27

you are taking a flat structure and turning it into tree, which is parsing, the difference is you are parsing a stream of datastructures and not a stream of characters

hiredman22:07:21

(require '[clojure.spec.alpha :as s]
         '[clojure.pprint :as pp])

(def ebook
  [{:class "indent" :content "ignore"}
   {:class "indent" :content "ignore"}
   {:class "START HERE" :content "Key Lessons"}
   {:class "indent" :content "ignore"}
   {:class "indent1" :content "ignore"}
   {:class "bl" :content "key lesson 1"}
   {:class "bl" :content "key lesson 2"}
   {:class "indent3" :content "ignore"}])

(s/def
  ::ebook-tree
  (s/and
    (s/cat
     :ignore (s/* (s/and map? #(= "ignore" (:content %))))
     :start (s/and map? #(= (:class %) "START HERE"))
     :body (s/* (s/alt
                 :ignore (s/and map? #(= "ignore" (:content %)))
                 :item (s/and map? #(= "bl" (:class %)))))
     :end (s/and map? #(= "indent3" (:class %))))
    (s/conformer
     (fn [{:keys [start body]}]
       (assoc start :children (for [[tag item] body
                                    :when (not= tag :ignore)]
                                item))))))

(pp/pprint
 (s/conform ::ebook-tree ebook))

denik22:07:46

Thanks, yes this works on this small example. I put days into trying to make this work with spec. Now imagine there are 1000's of different hashmaps with various content. In my experience, it’s hard to tell spec to ignore anything that doesn’t match without including everything.