any ideas on why I get errors creating a ThreadPoolExecutor?

(def t-pool (new ThreadPoolExecutor (int 8) (int 2) (long 60) TimeUnit/SECONDS (new LinkedBlockingQueue 100)))

3. Unhandled clojure.lang.Compiler$CompilerException
   Error compiling form-init7780702103892101766.clj at (36:13)
2. Caused by java.lang.reflect.InvocationTargetException
   (No message)
1. Caused by java.lang.IllegalArgumentException
   (No message)

https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadPoolExecutor.html

@beoliver The second argument is maximumPoolSize so it must be at least as large as the corePoolSize (the first argument).

I suspect you wanted (int 2) (int 8) instead?

Hello #clojure. I'm curious for your opinion on something. What thing(s) in Clojure would you choose to show someone without prior exposure to the language to demonstrate as much upfront practical value as possible?

Supposing your audience are experienced programmers familiar mostly with traditional imperative and object-oriented programming.

@mingp I think the first thing I would show would be REPL based development.

Aye, the REPL can be very compelling -- especially if you show a live, running process and connect a REPL into it and explore and modify it while it's running.

Depending on the background of your programming audience @mingp I think a demonstration of how easy it is to manipulate collections of data with map/filter/reduce. reducible-query in clojure.java.jdbc with a transducer pipeline, for example. Depending on the database and the configuration, you can stream very large record sets (that won't fit into memory) and still process them with transducers.

FWIW: What seemed to bowl over some devs I once knew was a single block of routes from compojure. The brevity got them interested.

Kind of a nice use case. Brevity. But also request/response as maps, thus highlighting the "it's all just data" idea.

For the working programmer, I usually demo the way clojure.java.jdbc allows integration of the Postgres geospatial and json queries and then add operators for json/geoquery using honeysql. I show "here's how you register serialization of these types to postgres, deserialization from postgres to my types, and this is how easy it is to query it with my custom operators. Put that together with compojure-api and I have a nice rest api console, validation for my endpoints, and arbitrarily complex SQL generated with very straight-forward functions (including demoing how easy transforms are when you have functions working with data). You can put it together on a couple pages and it's a real example that can be extended out in a useful way immediately.

I've mentioned this elsewhere, but I haven't found any language/ecosystem that is even close to as good at letting me clean data, generate complex SQL, and rapidly prototype well-documented apis. A lot of languages can let you build routes quickly, some even with beautiful documentation (django rest framework comes to mind with python). Almost every other solution gets insanely complex the minute I want to use database-specific features that invariably lead to string bashing.

A good exercise is to show the routes (say) and ask, 'Tell me what this code does' and see if they can get close.

The problem with a lot of Java frameworks (or even Django/Rails) is that it's super hard to figure things out without visiting dozens of files.

I imagine it's the same with SQL stuff (I just use the straight JDBC and strings).

In other words, it's not so much language nerd stuff, but all the get-it-done-fast so you can leave early stuff. And being able to figure out what others have done without also having to take anger management classes.

Thanks all. Appreciate the ideas, and will give it some more thought.

Hi, All. I'm fiddling with macros and and stumble with some behaviour which i think is strange.

(defmacro m1 [] (let [x (java.util.ArrayList.)] `~x))
(m1) ;; []
(type (m1)) ;; java.util.ArrayList

(defmacro m2 [] (let [x (org.joda.time.DateTime.)] `~x))
(m2) ;; #object[org.joda.time.DateTime 0x4011f484 "2017-10-23T13:05:15.179+07:00"]
(type (m2)) ;; CompilerException java.lang.RuntimeException: Can't embed object in code, maybe print-dup not defined: ...

@reedho i think the problem is that there is no literal representation of DateTime

(macroexpand-1 (m1)) <- this is the actual code that compiler is going to compile

i really don’t know what should (macroexpand-1 (m2)) look like

maybe you wanted your macro to spit out the actual (org.joda.time.DateTime.) ?

@keymone actually i'm working with larger macro to provide some inlining, above is minimalist example in that i guess some reader take place with the above macro expansion

It seems macro expanded values is serialized and then later need to be read again by reader

yes, and org.joda.time.DateTime is missing the kind of serialization that makes reader understand it’s reading a literal for that class

so, if it is something with reader, i wonder how [] can be brought back to java.util.ArrayList

(type (read-string "[]")) ;; => clojure.lang.PersistentVector

that is a good question

@reedho the problem is that you are embedding java objects in code, which is kind of a bad idea

@bronsa actually it came from the clj-time.core/*

still kind of a bad idea :)

Can anyone point me in the direction of a good example of best practice for implementing migrations on top of DynamoDB?

Regarding my question on macro above, actually the macro i'm trying to build is some thing like this:

(defmacro mk-route-check
  [routes]
  (let [dynamic-clauses
        (->> routes
             (mapcat
              (fn [{route :route
                    dows :dows}]
                [[
                  route
                  (if (= "DLY" dows)
                    '_
                    `(~'_ :guard
                      ~(->>
                        (clojure.string/replace-first dows #"^D" "")
                        (map (comp #(Long/parseLong %) str))
                        (into #{}))))
                  ]
                 true
                 ]
                )))]
    `(fn [route# dows#]
       (clojure.core.match/match
        [route# dows#]
        ~@dynamic-clauses
        :else false))))

This is okay, inlining the routes

(mk-route-check
 [{:route "CGK-DPS" :dows "DLY"}
  {:route "CKG-MDC" :dows "D123"}])

But i can not get it work if i want to load routes dynamically from somewhere, e.g.

(let [routes [{:route "CGK-DPS" :dows "DLY"}
              {:route "CKG-MDC" :dows "D123"}]]
  (mk-route-check routes))

you can't access run time values at compile time

So, its means there is now way to make it works?

I means, it can be made to work if i store the routes to root binding, and create macro based on that var

How does mk-route-check get called?

But then i can not make seamless reloading mechanism by providing only function

@carr0t some thing like:

(def __route-check nil)

(alter-var-root #'__route-check
                (constantly (mk-route-check
                             [{:route "CGK-DPS" :dows "DLY"}
                              {:route "CKG-MDC" :dows "D123"}])))

Macros run at compile time, so if you want to do something based on run-time variables your macro has to output forms which will do the right thing at run time

At which point the question may well be - why can't you just write a function instead of a macro?

I need macro so that i can make core.match clauses from some external data

I think like so 🙂

And that external data isn't available at runtime, it's only on the compile host? Can't it be in resources or something?

Yes, it is actually some edn got read from resources

So why can't that be done at runtime?

I want to achieve to e.g. provide simple function that could be used to reload the resource

e.g. (reload-route-check "resource-file")

then __route-check will refreshed with newest data

implementation below doesn't work:

(defn reload-route-check [resource-file]
  (let [routes
        (-> resource-file
            
            
            (java.io.PushbackReader.)
            clojure.edn/read)]
    (alter-var-root #'__route-check
                    (constantly (mk-route-check routes)))))

Is it bad form to query a db inside a map function if I need extra data for each item in collection? What are the alternatives?

@yonatanel yes, one alternative is to change the query with JOIN clause if possible

@paulbutcher Have you had a look at migratus? https://github.com/yogthos/migratus

I have, yes. And Ragtime and Joplin. I'm quite sure that all of these could work, but all of the examples are very SQL-focussed. I was hoping for some concrete experience/recommendations/best practice around DynamoDB 🙂

@reedho What if my api is against some service that only answers one by one? Which iteration fn is appropriate when each item requires out-of-process data and might fail?

Maybe for?

@yonatanel for that case, i will prefer to use mapv

@reedho Thanks!

@seancorfield 👍

a little bit of macro help? trying to construct a function that has the form (f xs (g (f xs (g (f xs ...))))) I have the following

(defmacro m1 [pool n]
  (if (= n 1)
    `(ft ~pool "did not block")
    `(ft ~pool (ft-deref (m1 ~pool (dec ~n))))))

for writing complicated macros, I like to first write FUNCTIONS that do "list of symbols" -> "list of symbols" ... and then afterwards, have the macro call the function

also, this particular macro looks like it can be written as a function of arguments f, g, xs

(defmacro m1 [pool n]
  (if (= n 1)
    `(ft ~pool "did not block")
    `(ft ~pool (ft-deref (m1 ~pool ~(dec n))))))

@beoliver last line ^

Curious to know why you'd want a macro for that though

@val_waeselynck I am playing with thread pools of (including ones of fixed sized)... if there are n threads and n+1 calls to deref then it will block. If I wrote it like this

(defn block-test [n pool]
  (loop [n n
         expr (ft pool "did not block")]
    (if (zero? n)
      expr
      (recur (dec n) (ft pool (ft-deref expr))))))

@val_waeselynck yeah I didn't need a macro

@beoliver good to know 🙂

Hi. Ihave this: [:a :b :c]. How do i convert it into this: [[:a] [:b] [:c]]? Thank you!

for each argument of your collection you want to call vector on the argument

So I have to use reduce? No simpler way?

(mapv vector [:a :b :c])

if you don’t need vector output, you could use map instead of mapv

does Clojure not :import public static final fields from Java?

what do you mean

nvm. figured it out

actually I didn’t. is there a way to pull static final constants from a Java class into a Clojure namespace?

no

(def foo Klass/foo)

@bronsa that's a bit confusing -- there's not a way, or (def foo Klass/foo) is the way?

you can’t pull static field directly in :import, you have to import class and then use (def foo Klass/foo)

NM, it was a colleague of mine looking for an answer, looks like he solved it.

@eggsyntax there's no way, but you can easily alias a static class field as a var using the above

Gotcha, that makes sense now. Thanks!

I have a question about sorted sequences - We have a large sequence/vector/set (multiple thousands of entries). We need it to be ordered, and we need to add to and remove from it. When we add we know the value of the thing we'd be sorting on (obviously), but when we remove we do not

How would you go about doing that 😉

Our initial naive version of a vector is hugely slow when we reconstruct it because we have to sort everything

A sorted set looked like it would work, but that converts to a regular set when we remove things from it

> When we add we know the value of the thing we'd be sorting on (obviously), but when we remove we do not can you explain this more?

Basically, we're sorting on a timestamp

Each item also has an id

When we remove, we want to remove all things with id XXX (matching the thing we're about to insert) regardless of the timestamp they have

oh i see

Or actually, we're sorting on timestamp and ID, because it is possible for 2 items with different IDs to have the same timestamp

is it onerous to keep two copies of the collection?

one hashed on ID, the other a binary tree by timestamp with collections of things at timestamp?

Hmm

look up by id , get timestamp, and then remove from binary tree?

That could work. It feels a bit nasty, but could work

there's clojure.data.avl or something similar that would keep it balanced for you, not have to reinvint the wheel

storing transit data as jsonb in postgres - has anyone been burned by this?

How do I create a map, in which the order is retained like in java.util.LinkedHashMap? I need vals to return the values in the order in which they were assoc'ed to the map...

you could add a timestamp to the values and use a sorted map? https://clojuredocs.org/clojure.core/sorted-map-by

@carr0t Do you need fast nth support on the sorted set? Or you just want to iterate it sorted by your values?

Well, we want to be able to periodically (like, every second) pull from it all events with timestamp < 'now', and push them onto our job queue

So that means you'll be taking from the beginning until somewhere.

Yeah

Then this should fulfil your requirements: https://github.com/clojure/data.priority-map

A question about refs. Assuming that I have a data structure that I wish to share between multiple threads, but I want to both read and update in one single atomic action, do either of the following two examples capture this? I am unsure of the scope of dosync. As far as I am aware this (and refs in general) are used for atomic transactions using more than one state:

(def mah-data (ref [1 2 3 4 5]))

(defn mah-update-1 [data-ref]
  (dosync
   (when-let [a-value-that-no-other-thread-can-have (-> data-ref deref peek)]
     (alter data-ref pop)
     (println "value is" a-value-that-no-other-thread-can-have))))

(defn mah-update-1 [data-ref]
  (dosync
   (when-let [a-value-that-no-other-thread-can-have (-> data-ref deref peek)]
     (alter data-ref #(if (empty? %)
                        identity
                        (pop %)))
     (println "value is" a-value-that-no-other-thread-can-have))))

Intuitively I feel that I need to be requiring the lock before the deref, but I am not sure if this is the case using dosync

If you only have one piece of state @beoliver you should be able to use atom and avoid dosync etc.

ref is useful when you have two separate stateful things that you want to update together.

(and, thus, ref is rarely needed or used)

@beoliver You can use a "trick" like this

(let [state (atom [1 2 3])
      x (meta (swap! state
                     (fn [x]
                       (let [removed (peek x)]
                         (with-meta (pop x) {:rem removed})))))]
  [x @state])

@rauh is there any info on he trick? Is this the only way? If so, I am guessing that this must be an operation that is not considered idiomatic 😄

@beoliver You don't have to, you can also avoid meta and just keep your "last removed item" in the state, but then whenever you want to use state you have to first get rid of the "last removed item"

(let [state (atom [nil [1 2 3]])
      [lr] (swap! state (fn [[_ x]] [(peek x) (pop x)]))]
  [lr (second @state)])

would

(defn mah-update-3 [data-atom]
  (let [p (promise)]
    (swap! data-atom (fn [x] (deliver p (peek x)) (pop x)))
    (println "value is" @p)))

@beoliver That's not going to work

yeah i see

cant call it multiple times

If swap! decides to re-run your fn, then your promise will still have the first (wrong) value

exactly

You could certainly use another atom and reset that one. That'd work

@rauh 👍

hello

the require function is not working within the proxy function. Does anybody know why?

gonna need a little more info to help @nick319

@ghadi thanks for your help. I am calling require to include a library within a function used by proxy

keep in mind require executes at runtime, not compile time... Stuff like this cannot work:

(let [...]
  (require 'foo.bar)
  (foo.bar/something...))

@ghadi here is a code snippet:

(proxy [VisionWindowListener] [] (windowOpened [win] (require '[clojuretest.windows.main-window :as script])

When I do that the library is not found

try to find a smaller error. Does running (require '[clojuretest.windows.main-window :as script]) work in your REPL?

I am wondering if it is because VisionWindowListener is using a different classloader than the library that I am trying to load

Yes

calling require right above proxy works. For example:

(proxy [VisionWindowListener] [] (windowOpened [win] (require '[clojuretest.windows.main-window :as script])

(require '[clojuretest.windows.main-window :as script])

(require '[clojuretest.windows.main-window :as script]) (proxy [VisionWindowListener] [] (windowOpened [win]

but I want to call require within a function defined in proxy

You only need to call require once at the top of your namespace, unless you are dynamically loading code

I understand. I am dynamically loading code, which is why I am calling it within a function that is within proxy

the example you're providing is not dynamic.

That is true, first I am trying to get a non-dynamic version working before I make it dynamic

So what are you going to do after you require the dynamic namespace?

then I am going to call function from the dynamic namespace

Does functions defined in proxy use the class loader of the interface that is being extended or is the classloader used the same one that clojure uses?

Because the interface I am implementing with proxy uses a different classloader than my clojure code

So I am wondering if that is the issue --- wondering if my proxy uses a different classloader than my regular code

what error are you getting

Is it legal to use require inside proxy function definitions? because everywhere else I use require, it works

I will get the error message

the problem is the scenario i described above

https://clojurians.slack.com/archives/C03S1KBA2/p1508791526000138

That will not compile ^

This is my error:

java.io.FileNotFoundException: Could not locate clojuretest/windows/main_window__init.class or clojuretest/windows/main_window.clj on classpath. Please check that namespaces with dashes use underscores in the Clojure file name. at clojure.lang.RT.load(RT.java:456) at clojure.lang.RT.load(RT.java:419) at clojure.core$load$fn__5677.invoke(core.clj:5893) at clojure.core$load.invokeStatic(core.clj:5892) at clojure.core$load.doInvoke(core.clj:5876) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.core$load_one.invokeStatic(core.clj:5697) at clojure.core$load_one.invoke(core.clj:5692) at clojure.core$load_lib$fn__5626.invoke(core.clj:5737) at clojure.core$load_lib.invokeStatic(core.clj:5736) at clojure.core$load_lib.doInvoke(core.clj:5717) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:648) at clojure.core$load_libs.invokeStatic(core.clj:5774) at clojure.core$load_libs.doInvoke(core.clj:5758) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:648) at clojure.core$require.invokeStatic(core.clj:5796) at clojure.core$require.doInvoke(core.clj:5796) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojuretest.core$create_window_listener$fn__13.invoke(core.clj:22) at clojuretest.core.proxy$java.lang.Object$VisionWindowListener$9dcff2dc.windowOpened(Unknown Source) at com.inductiveautomation.factorypmi.application.runtime.ClientContextImpl$1.windowOpened(ClientContextImpl.java:166) at com.inductiveautomation.factorypmi.application.FPMIApp.fireWindowEvent(FPMIApp.java:1197) at com.inductiveautomation.factorypmi.application.FPMIApp$InternalFrameEventRelay.internalFrameOpened(FPMIApp.java:1319) at javax.swing.JInternalFrame.fireInternalFrameEvent(Unknown Source) at javax.swing.JInternalFrame.show(Unknown Source) at java.awt.Component.show(Unknown Source) at java.awt.Component.setVisible(Unknown Source) at javax.swing.JComponent.setVisible(Unknown Source) at com.inductiveautomation.factorypmi.application.FPMIApp$RuntimeWindowOpener.openWindow(FPMIApp.java:1774) at com.inductiveautomation.factorypmi.application.FPMIApp.openWindow(FPMIApp.java:1014) at com.inductiveautomation.factorypmi.application.script.builtin.SecurityUtilities._tempReOpen(SecurityUtilities.java:230) at com.inductiveautomation.factorypmi.application.runtime.UpdateProjectPane.uninstall(UpdateProjectPane.java:103) at com.inductiveautomation.factorypmi.application.runtime.UpdateProjectPane$1.stepsFinished(UpdateProjectPane.java:91) at com.inductiveautomation.ignition.client.launch.AbstractStepRunner$1.run(AbstractStepRunner.java:40) at java.awt.event.InvocationEvent.dispatch(Unknown Source) at java.awt.EventQueue.dispatchEventImpl(Unknown Source) at java.awt.EventQueue.access$500(Unknown Source) at java.awt.EventQueue$3.run(Unknown Source) at java.awt.EventQueue$3.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source) at java.awt.EventQueue.dispatchEvent(Unknown Source) at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source) at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source) at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source) at java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.awt.EventDispatchThread.run(Unknown Source)

because the compiler doesn't know what foo.bar/something is until the function runs...

(require 'clojuretest.windows.main-window :reload) does that work in a REPL?

I just tried it, yes

if you want to eliminate proxy from suspicion, you can move the body of the proxy method to an ordinary defn and try it there

I did that, it works in a regular defn

It just doesn't work under proxy

Also does not work under reify

I looked at your example:

(let [...] (require 'foo.bar) (foo.bar/something...))

Why can't that run at runtime and work?

because the compiler errors if it can’t find foo.bar - you need a layer of indirection (eg. resolve) around the lookup of something to make that work

oh so if foo.bar is there then it will work

if it’s required in your ns form

or, if you already ran the code once so it already loaded it earlier…

this is affectionately called the "gilardi scenario"

https://technomancy.us/143

a small example of require works fine in reify ... I'm wondering if something else is at play

(defn bar [] (reify clojure.lang.IReduceInit (reduce [_ f init] (require 'foo))))

;;; foo.clj
(ns foo)
(println "hey")

What if you try to reify an interface that uses a different classloader than clojure

In that code example you just gave does require use clojures classloader or does it use the interfaces classloader?

clojure's classloader -- sorry I can't work up a larger classloader example at the moment

@ghadi thanks

@ghadi yes, but if clojure's class loader delegates to its parent class loader and if the parent class loader has the class then the parent class loader will then be used to load the classes in the class, and not clojure's class loader. I'm thinking that might be the problem. I am doing an experiment to find out.

I know we have https://github.com/clojure/test.check in clojure, which is great. Does the reverse exists? For example: I was trying to isolate an error in a biggish generated EDN/JSON structure. I have a unit test that fails with said structure with a given error. Would there be a way to remove parts of the structure to make is smaller to read, while keeping the same error? It seems it is a general debugging process I follow quite often (remove parts until it is small enough to understand it).

@nha I haven't seen such a thing. I suppose it'd be theoretically possible to reuse test.check's shrink-loop if you wrote the code for doing the shrinking. but that's most of the work anyhow.

@ghadi I solved the problem with using require in proxy and reify

@ghadi The code that created the proxy object runs in a different thread than the code that calls the methods on the proxy. In the thread that calls the proxy methods the context classloader was not set to the classloader that has clojure. So simply setting the context classloader to clojure on the thread that calls the proxy methods fixed the problem. Thanks for looking into this with me.

@nha https://github.com/pjstadig/humane-test-output might be what you're looking for, but I don't have enough information about your particular situation to say for sure? If you have that loaded and try to compare to maps, then it will produce a diff if they don't equal each other.