Fork me on GitHub
#clojure
<
2019-07-16
>
caleb.macdonaldblack00:07:52

I have this for loop causing a java.io.IOException: File name too long exception when running lein uberjar any way to get around this? I think I could probably create the combinations I need another way however I'd like to understand this a little more. for is a macro and it seems to be creating some long symbol names when expanding this deeply nested for loop.

caleb.macdonaldblack00:07:53

my namespace is also 39 characters long which also be contributing to this error

andy.fingerhut00:07:41

I suspect you are hitting this reported issue: https://clojure.atlassian.net/browse/CLJ-1852

andy.fingerhut00:07:18

Internally generated function names with for get longer as the number of nested loops in the for increases, and you have quite a few there.

andy.fingerhut00:07:16

As far as workarounds go, one would be to use a library like math.combinatorics to generate all of the combinations in a flat list, and iterate through those.

andy.fingerhut00:07:57

A really quick thing to try would be to put the values of min and time-delim into a let surrounding that for, since they never change.

caleb.macdonaldblack00:07:08

@andy.fingerhut Thanks for that. Good idea with the let block, for the time being that will work nicely. Cheers

andy.fingerhut02:07:32

well (-> m1 :cells [1 1]}) gives you {:north :foo ...}

andy.fingerhut02:07:09

Once you get to any map, (key some-map) gives you a sequence of only its keys, and (vals some-map) a sequence of only its values.

Amanu07:07:45

Type keys?

andy.fingerhut07:07:10

Can you say a little more in that question to help me understand what you are asking?

andy.fingerhut07:07:10

Oh, if you are asking me whether I should have said (keys some-map), then yes, I should have.

Amanu12:07:06

Sorry, i meant to say typo* That makes more sense

pinkfrog03:07:56

can i create an alias to a package?

pinkfrog03:07:18

for example, i want to refer to langohr.core as myfoobar.core

pinkfrog03:07:28

basically, replacing langohr with myfoobar

andy.fingerhut03:07:13

I have never done this before, but just tried out (:require [clojure.pprint :as foo.bar]) followed by (foo.bar/pprint ...) and it seemed to work.

andy.fingerhut03:07:35

I don't know of any way to globally make foo and alias for the clojure part of clojure.pprint simultaneously with clojure.java.browse, etc. The :as method lets you do it one namespace at a time.

pinkfrog04:07:29

@andy.fingerhut yup. i need making foo referring to clojure.

pinkfrog04:07:20

the prefixed naming makes the code really cluttered.

pinkfrog04:07:25

e.g., lc, lch, lq, lb

hiredman04:07:42

It does not make it cluttered, it makes it clearer at a glance, I suggest readjusting your esthetic sensibilities to accept it

16
pinkfrog10:07:54

hi. would be appreciated if you could be explicit about the naming convention.

pinkfrog10:07:37

e.g., is the naming be like, for some.package.namespace be referred as sp.namespace? or sp.n? or ?

emccue14:07:10

I usually do some.package.namespace :as namespace

emccue14:07:28

clojure.data.json :as json

emccue14:07:38

(json/read ...)

vemv15:07:39

We have adopted https://stuartsierra.com/2015/05/10/clojure-namespace-aliases team-wide at work (even having a private linter for it) and haven't looked back so far

Drew Verlee17:07:25

How we view the id/name is largely a personal preference right? It can make it more clear or it can make it more cluttered. By aliasing at all we admit that the full name is undesirable to work with in context, but the cutoff is always arbitrary and usually just a convention

[clojure.spec.alpha :as s]
Is "ok" for spec, but not for some other lib, there you should use the full name! I'm hopeful that in the future will have ways to easily toggle between views without having to beat our head against the wall trying to deiced which is best.

Drew Verlee17:07:09

but yea, for the time being. What hired and emccue said 🙂

liath21:07:34

Idk if this is the right place to ask but could I trouble someone to review my code? https://github.com/brunoV/throttler/pull/12 The libraries creator isn't active anymore due to health reasons and I'm not terribly proficient with clojure or Java.

noisesmith21:07:43

a fundamental misconception I see right away: calling timeout in core.async does not use a thread

noisesmith21:07:15

the premise of go blocks is to have green threads that are lightweight, and using <! to access a timeout does not sleep any thread - it creates a callback using a timer

noisesmith21:07:29

it's shared by all timeouts - but it is a single thread

noisesmith21:07:52

I might have been misreading though - it seems like this is more about the timer accuracy

liath21:07:12

Yeah, the original author seemed to prefer to reuse what was already available so I feel bad using more resources

noisesmith21:07:35

I know this is shared with the original, but taking the reciprocal of rate-ns in chan-throttler* is counterintuitive

noisesmith22:07:44

instead of calling double in your sleep time calculation, you could replace :nanosecond 1 with :nanosecond 1.0 in unit->ns

hiredman22:07:30

the scheduledexecutor version will use threads and eventually run out of them

4
noisesmith22:07:30

I think the doc needs fixing - currently it claims no new threads are created (mostly true), your edit means each throttled thread uses the executor thread pool (almost true but less so)

hiredman22:07:17

because they will be blocked trying to write to a channel that isn't being read from (unless the rate limited function is being called)

noisesmith22:07:05

isn't the dropping buffer helpful there?

noisesmith22:07:22

it means a max of buffer-size threads are being wasted

noisesmith22:07:28

(not ideal but better)

hiredman22:07:38

you could get rid of threads/core.async etc all together and just keep an atom and track the time

noisesmith22:07:52

also using put! instead of >!! might be an improvement

liath22:07:57

I assumed the newSingleThreadScheduledExecutor bit meant that it would create a pool of 1 thread.

noisesmith22:07:03

oh - and blocking that thread until something tries to execute would be fine for the internal logic anyway

noisesmith22:07:19

but that also means if you have two throttled functions a and b, a is halted altogether if b doesn't get called

noisesmith22:07:22

and visa versa

noisesmith22:07:52

since you share one scheduler between all throttled functions, and the >!! doesn't return until someone tries to execute a function

noisesmith22:07:32

I guess if your buffer sizes are all > than the sum total of throttled functions you're OK - but that is very brittle and weird

noisesmith22:07:18

maybe I still have this wrong...

liath22:07:35

With the dropping-buffer would there ever be a reason for the executor to be blocked?

noisesmith22:07:20

no, you're right, with the dropping buffer you just get lag time introduced by the number of throttled channels being used (potentially), but it won't lock up

liath22:07:06

I think maybe this is why core.async/timeout buckets requests.

liath22:07:12

I'll have to create a test case with multiple timers to see how bad it behaves

hiredman22:07:39

you could do something like

(let [max-tokens 10
      token-rate 100
      x (atom {:tokens 0 :last-tokens (System/currentTimeMillis)})]
  (fn [f arg]
    (loop []
      (let [{:keys [tokens last-tokens] :as m} @x
            now (System/currentTimeMillis)]
        (if (pos? tokens)
          (if (compare-and-set! x m (assoc m :tokens (dec tokens)))
            (f arg)
            (recur))
          (if (>= (- now last-tokens) token-rate)
            (do
              (compare-and-set! x m (assoc m :tokens (min max-tokens (+ tokens (/ (- now last-tokens) r))) :last-tokens now))
              (recur))
            (do
              (Thread/sleep token-rate)
              (recur))))))))

hiredman22:07:00

you just have to watch out for time going backwards between threads

hiredman22:07:38

with that code you don't have other threads managing the buckets and tokens, and tokens aren't pointers kept around in a channel's buffer

hiredman22:07:59

I hadn't really looked at the rest of the library, I guess it doesn't just throttle function calls

liath22:07:32

Could be applicable either way but avoiding thread/sleep was another design goal

hiredman22:07:47

shrug blocking on a channel that yields a value in some amount of time is the same thing

liath22:07:53

Is it? I was under the impression that other things could be going on while waiting for a channel but thread/sleep was like a pause-the-world button

hiredman22:07:56

it is different inside a go block using >! and <!, but the function call throttling code is not in a go block and using >!! and <!! which block the thread

noisesmith21:07:46

there is a #code-reviews channel also

liath21:07:22

Oh, thank you!

liath21:07:59

Er, now I feel like if I delete this and repost it there I'll leave you hanging

Eric Ervin21:07:46

doesn't hurt to leave it

vlaaad22:07:05

I was thinking about formatting data structures and came up with this idea:

{:fx/type :stage
 :showing true
 :always-on-top true
 :style :transparent
 :scene
   {:fx/type :scene
    :fill :transparent
    :stylesheets
      #{"styles.css"}
    :root
      {:fx/type :v-box
       :children
         [{:fx/type :label
           :effect
             {:fx/type :drop-shadow
              :radius 1
              :offset-y 2}
           :tooltip
             {:fx/type :tooltip
              :text "I am a tooltip!"}
           :text "Hi! What's your name?"}
          {:fx/type :text-field}]}}}
What do you think?

Joe Lane22:07:01

@vlaaad I don't want to assume, but did you mean to put this on #cljfx ?

vlaaad22:07:33

no, this is an idea for data structure formatting, actual contents are incidental 🙂

andy.fingerhut22:07:36

If you are talking about the method of breaking up into lines and inserting white space, then I don't have the corresponding clojure.pprint/pprint output at the tip of my brain -- is your idea significantly different than what it does?

vlaaad22:07:19

corresponding pprint output:

{:fx/type :stage,
 :showing true,
 :always-on-top true,
 :style :transparent,
 :scene
 {:fx/type :scene,
  :fill :transparent,
  :stylesheets #{"styles.css"},
  :root
  {:fx/type :v-box,
   :children
   [{:fx/type :label,
     :effect {:fx/type :drop-shadow, :radius 1, :offset-y 2},
     :tooltip {:fx/type :tooltip, :text "I am a tooltip!"},
     :text "Hi! What's your name?"}
    #:fx{:type :text-field}]}}}

seancorfield22:07:49

@vlaaad I find your first example a bit misleading to read since indentation to me implied substructure so I find it odd to read keys with their value indented when it isn't nested. If you see what I mean?

vlaaad22:07:11

But there is substructure in indentation?..

seancorfield22:07:16

And I guess my first question would be "why format data differently from code?"

vlaaad22:07:50

Too much horizontal space wasted with deep nesting

seancorfield22:07:06

But your suggested format has more horizontal space since it indents more that pprint would...?

vlaaad07:07:04

but it's not?

{:long-key [some-coll]}
uses less horizontal space than
{:long-key
   [some-coll]}
, because value does not have to pay for key's length