Fork me on GitHub
#clojure
<
2020-09-28
>
jumar03:09:34

A question regarding lazy-seq. I'm not sure why I cannot use conj instead of cons here:

(defn simple-range [i limit]
  (lazy-seq
   ;; nil returned by when will terminate the range construction
   (when (< i limit)
     (print ".")
     ;; notice you cannot use `conj` because get StackOverflowError in the `take 10` below
     (cons i (simple-range (inc i) limit)))))
(take 10 (simple-range 1 1e20))
;; => (1 2 3 4 5 6 7 8 9 10)

;; conj doesn't work
(defn simple-range2 [i limit]
  (lazy-seq
   ;; nil returned by when will terminate the range construction
   (when (< i limit)
     (print ".")
     ;; notice you cannot use `conj` because get StackOverflowError in the `take 10` below
     (conj (simple-range2 (inc i) limit) i))))
;; StackOverflowError
#_(take 10 (simple-range2 1 1e20))

phronmophobic03:09:22

🤷 works for me

my.ns> (defn simple-range [i limit]
  (lazy-seq
   ;; nil returned by when will terminate the range construction
   (when (< i limit)
     (print ".")
     ;; notice you cannot use `conj` because get StackOverflowError in the `take 10` below
     (conj  (simple-range (inc i) limit) i))))
#'my.ns/simple-range
my.ns> (take 10 (simple-range 0 10))
..........(0 1 2 3 4 5 6 7 8 9)

phronmophobic04:09:26

oops, nvmd. it overflows if the limit is really high

seancorfield04:09:42

conj is implemented as coll.cons(x) so it's going to require that collection to be unwound far enough to cons that last element on.

seancorfield04:09:35

conj is a collection function, rather than a sequence function -- and it's easier to think of those as eager whereas sequence functions are usually lazy (not accurate, but a reasonable heuristic)

Takis_10:09:41

Hello 🙂 i have a dependency from github, i used https://github.com/reifyhealth/lein-git-down ,using the code from the Example of the project readme

Takis_11:09:11

classes are found,but it isnt working normally,i think exceptions happen,any help? or alternative way to do it ?

Takis_11:09:34

i am using leiningen

p-himik11:09:38

What exception? What dependency?

Takis_11:09:01

i don't know it is handled from inside the code,the way i did it using that plugin , is the common way to do it using leiningen ?

Takis_11:09:48

i try to use this

Takis_11:09:11

but from github not from maven

p-himik11:09:04

But you say that it isn't working normally - how do you know that it isn't? What are the symptoms?

Takis_11:09:26

its a music player,gets audio from youtube,but produces TrackExceptionEvent , when the maven same version doesnt

Takis_11:09:41

but its working in general,other things from that code works,its the first time i used github dependency,and i dont know if lein-git-down is the right way to do it

Takis_11:09:10

if the plugin is working ok,i guess its my fault somewhere

p-himik11:09:02

Moving to a thread to avoid spamming the main channel. By "same version" you mean 1.3.50 and 60441fc5?

p-himik11:09:38

And does the TrackExceptionEvent have any stack trace that you could share?

Takis_12:09:08

On maven i use this

Takis_12:09:31

on github the same is this i think

Takis_12:09:18

my project.clj looks like

Takis_12:09:03

(defproject test-project "0.1.0"
    :description "A test project"

    :plugins [[reifyhealth/lein-git-down "0.3.7"]]
    :middleware [lein-git-down.plugin/inject-properties]
    :dependencies [[lavaplayer "60441fc5a4f58e80a5466fa36b9366fb8501a8f5"]]
    :repositories [["jcenter-bintray" ""]
                 ["bintray-rotate" ""]
                 ["public-github" {:url ""}]
                 ["private-github" {:url "" :protocol :ssh}]]
    :git-down {lavaplayer {:coordinates sedmelluq/lavaplayer}})

Takis_12:09:39

i dont have the stack trace,i dont know how to get it, i get an Event object,that is made from that Exception handler i guess

p-himik12:09:53

When you see that TrackExceptionEvent string somewhere - can you share a screenshot?

p-himik12:09:34

The probability of it being an issue with lein-git-down is quite high IMO because lavaplayer has not only Java code that can already be tricky with Git dependencies, but also native C code that's much more trickier.

Takis_12:09:00

still i didnt found the stacktrace,but i found the Exception

Takis_12:09:03

com.sedmelluq.discord.lavaplayer.player.event.TrackExceptionEvent "com.sedmelluq.discord.lavaplayer.tools.FriendlyException: Something broke when playing the track."

Takis_12:09:42

its the first time i used github dependency,how people do it?

Takis_12:09:29

i saw the deps.edn new cli , i dont know , but never used it,only leiningen i used

p-himik12:09:11

"FriendlyException", ha! - with a completely unfriendly message. Says absolutely nothing.

Takis_12:09:16

thank you for helping me,its not so important thing, but its usefull to know how to use github dependencies in general

p-himik12:09:10

Git dependencies are hard. Deps CLI will not help you here as it's designed to work with the dependencies that don't need any compilation - you can't even have Java files if you don't have precompiled class files for them.

Takis_12:09:16

how to use a java project from github? even in local way

p-himik12:09:41

I couldn't find any build instructions for lavaplayer. Potentially, the maintainers can do anything before they publish the jar to the Maven repository. But even if there were some build instructions, there's no tool out there that would be able to follow them automatically.

Takis_12:09:06

anyway its ok , i will leave it for now , thank you for helping me

Takis_12:09:44

i will check it again sometime 🙂

p-himik12:09:44

With regular Java and regular build steps, such tools like lein-git-down should indeed help. You can also have such a dependency not as a regular jar dependency but as vendored sources in your project - just as if it was part of your project. But don't do that unless you need to modify the sources of the dependency.

p-himik12:09:29

No problem, HPH.

Takis_12:09:54

yeah lein-git-down seems to work,maybe that lavaplayer is exception , thank you 🙂

victorb15:09:48

anyone got any dns libraries laying around/to recommend? Been using https://github.com/brweber2/clj-dns but recently started using java 11 and seems *sun*.*net*.*spi*.*nameservice*.*NameServiceDescriptor* doesn't exist since Java 9, so no longer works. Interested in doing lookups of TXT records only

victorb15:09:32

Thanks @UTQEPUEH4, yeah, by the description it seems like it'll do the trick, trying to figure out how to actually use it (never done any serious Java programming)

victorb15:09:16

seems to have the most cumbersome API I've seen in the Java world, but I haven't seen too much of it so.

jsn15:09:44

(-> (javax.naming.directory.InitialDirContext.) (.getAttributes "") (.get "TXT")) seems to work for me

jsn15:09:30

or even (-> (javax.naming.directory.InitialDirContext.) (.getAttributes "dns:///ice-9.eu") (.get "TXT")) , actually

victorb15:09:09

It does indeed. I had just got half-way through the docs and even further from understanding. Still not sure how naming.directory comes to do DNS resolution, would expect that to be in some of the networking parts of Java

victorb15:09:19

anyways, you're a hero @UTQEPUEH4, thanks a lot for helping out

victorb16:09:51

for future reference, lazy wrapper to collect the record values as strings:

(defn txt-records [domain]
  (let [results (atom [])
        query-res
        (-> (javax.naming.directory.InitialDirContext.)
            (.getAttributes (str "" domain))
            (.get "TXT"))
        enum (.getAll query-res)]
    (while (.hasMore enum)
      (swap! results conj (.next enum)))
    @results))

jsn18:09:41

(-> (javax.naming.directory.InitialDirContext.) (.getAttributes "") (.get "txt") .getAll enumeration-seq) ?

emccue20:09:19

be careful with anything under the javax package

emccue20:09:59

I think there is actually a licensing violation if you ship software using that package name with classes not provided by oracle (or something like that)

victorb20:09:56

eh, interesting and very annoying. Thanks for the headsup @emccue

jsn21:09:58

Why is that annoying, why would you want to ship any class names under javax. not provided by Oracle (and how does that relate to the dns lookup code above, exactly)?

jsn21:09:14

@UEJ5FMR6K did you see that you don't need atom and the imperative stuff?

vlaaad20:09:10

hmm, ho do I do map if every item in a result coll depends on a previous one?

vlaaad21:09:28

hmm, looks like what I need

vlaaad21:09:57

thank you!

🙂 3
Saikyun09:10:20

aha, very cool. thanks!

vlaaad20:09:33

e.g. there is a "state" that is accumulated while mapping

vlaaad20:09:53

but the result is always the same length as initial coll

vlaaad20:09:09

I was thinking there could be a lazy way to do that

vlaaad20:09:54

but iterate is on a single value, and it's infinite

vlaaad20:09:30

while my result is finite

vlaaad20:09:46

I'll just use reduce I guess

ghadi20:09:14

iteration

vlaaad20:09:26

but there is no side effects in my case

ghadi20:09:20

then iterate will work fine

vlaaad20:09:23

it's a map with some intermediate state calculated from inputs that has to be updated and read on each step

vlaaad20:09:01

I just find it a bit strange that the thing I want to think of as a "map with accumulated state" turns into this big chunk of code:

(:acc
  (reduce
    (fn [{:keys [state acc]} [dt [event n]]]
      (let [state (case event
                    :price (assoc state :last-price n)
                    :div (update state :stock + (* (:stock state) (/ n (:last-price state)))))]
        {:state state
         :acc (conj acc [dt state])}))
    {:state {:stock 1 :last-price 0}
     :acc []}
    (sorted-map ;; example input
      0 [:price 10]
      1 [:div 1]
      2 [:price 20]
      3 [:div 2])))

jjttjj21:09:46

If you're just interested in the previous item you could do

(defn map-prev
  ([f]
   (fn [rf]
     (let [vold (volatile! nil)]
       (fn
         ([] (rf))
         ([result] (rf result))
         ([result x]
          (let [old @vold
                new
                (if (some? old)
                  (vswap! vold f x)
                  (vreset! vold x))]
            (rf result new)))))))
  ([f xs]
   (map f xs (rest xs))))

jjttjj21:09:02

(I've been focusing more on the transducer versions)

vlaaad21:09:24

Yeah, using mutable volatile is a fine option for single-threaded processing!

jjttjj21:09:30

note with that map-prev nil is not a valid value, and it just sets the previous state the first time it sees a value.

jjttjj21:09:50

but otherwise it's like map but the function you pass it just takes [old new]

emccue20:09:16

(defn running-total-of-price [price-points]
  (loop [price-points (seq price-points)
         state {:stock 1 :last-price 0}
         price-history []]
    (if (empty? price-points)
      {:state state :acc price-history}
      (let [[dt [event n]] (first price-points)
            new-state (case event
                         :price (assoc state :last-price n)
                         :div (update state 
                                      :stock
                                      +
                                      (* (:stock state) 
                                         (/ n (:last-price state)))))]
        (recur (rest price-points)
                new-state
                (conj price-history [dt state]))))))

emccue20:09:55

the way we learned map in school was just "recursion with linked lists"

emccue20:09:30

so writing if empty? ... else (recur ... ) stuff is maybe more readable to me than it should be

emccue20:09:45

but at least I can parse it better than the equivalent reduce

emccue20:09:15

I don't know if thats helpful or not

vlaaad21:09:12

agree on readability @emccue

jeaye22:09:24

Is there any recommended way to get jsonista to losslessly encode keyword values so this returns equivalent data?

user=> (-> (j/write-value-as-string {:system/status :status/good} json-mapper) (j/read-value json-mapper))
#:system{:status "status/good"}
Transit+json does the job fine, but it's much slower.

emccue22:09:35

@jeaye conceptually, for it to work regardless of your data format you need to "tag" keywords where-ever they appear

emccue22:09:45

(separate from the library)

emccue23:09:17

but also because keywords can appear as keys in maps, you need to manipulate the json structure quite a bit to losslessly encode and decode

emccue23:09:11

What you could do though, if you are feeling kinda lazy, is just put a prefix

emccue23:09:22

so make a custom object mapper to give to the library

emccue23:09:28

that when it encounters a keyword

emccue23:09:32

instead of mapping it to

emccue23:09:43

:some-keyword => "some-keyword"

emccue23:09:13

:some-keyword => "💩some-keyword"

emccue23:09:24

and just look for the prefix when round tripping

emccue23:09:18

as a consequence you might need to filter the poop emoji out from user input

emccue23:09:53

(transit gives the "right" answer, but there are tons of hacky ones like this)

jeaye23:09:57

Yeah, I've considered that approach. I was hoping for something less clunky, but it may not exist.

emccue23:09:32

yeah if you want this

emccue23:09:02

{"a" :abc :def "hijklmnop"}

emccue23:09:40

to round trip correctly, you would have better luck just figuring out how to speed up the transit library to match your needs

emccue23:09:08

well, the issue with maps is just that they always require string keys in the json repr

emccue23:09:32

so in your own does-kinda-the-same-as-transit-but-not you can

emccue23:09:31

:a => ["k", "a"]
"a" => ["s", "a"]
{:a 2} => ["m", ["k", "a"], 2]
{:a 2} => ["m", ["s", "a"], 2]
[:a 2] => ["v", ["k", "a"], 2]
#{:a} => ["@", ["k", "a"]]

emccue23:09:51

always box everything

emccue23:09:23

or you can just use xml

emccue23:09:23

<map>
  <entry> 
    <key> <keyword value="a"></string> </key>
    <value> <number value=3></number> </value>
  </entry>
</map>

emccue23:09:42

or just use edn and give up

cfleming23:09:07

This is driving me nuts - either I’m doing something very stupid and I’m too tired to see it, or I’m misunderstanding something subtle:

(def common-errors #{{:unexpected :except, :expected #{":only" ":rename" ":exclude" :eof}}
                     {:unexpected :include-macros, :expected #{":as" ":only" ":refer" ":rename" ":exclude" :eof}}})
=> #'user/common-errors
(->> (take 5 errors)
     (remove (fn [error]
               (every? #(do (prn %)
                            (println (contains? common-errors %))
                            (contains? common-errors %))
                       (errors-from (:new error)))))
     (count))
{:unexpected :default, :expected #{":as" "sequential" ":only" ":refer" "symbol" ":rename" ":exclude" :eof}}
false
{:unexpected :except, :expected #{":only" ":rename" ":exclude" :eof}}
false
{:unexpected duration-inst, :expected #{"sequential"}}
false
{:unexpected :except, :expected #{":as" "sequential" ":only" ":refer" "symbol" ":rename" ":exclude" :eof}}
false
{:unexpected :include-macros, :expected #{":as" ":only" ":refer" ":rename" ":exclude" :eof}}
false
=> 5
(contains? common-errors {:unexpected :except, :expected #{":only" ":rename" ":exclude" :eof}})
=> true

cfleming23:09:14

I have a common-errors set of error forms. I’m trying to filter those out of my list of errors. When I test a form manually against the set contains? returns true, but inside the remove it doesn’t.

cfleming23:09:25

I’ve copy-pasted the manual test from the prned version, so it should be the same structure. There’s no metadata involved either.

noisesmith23:09:08

when all else fails I like to capture the actual data, so I can check the types (eg. a symbol with a colon that prints like a keyword) - though it looks like you have that data and could just jump into checking that the printed values and assumed types match up

dpsutton23:09:28

the second one in this list should have matched true? maybe chop down the the common-errors to just the first one and change it to equality rather than contains? and perhaps run it through data/diff. Is there any chance there's some shadowing going on here so common-errors isn't what you think it is?