Fork me on GitHub
#beginners
<
2018-11-16
>
zlrth00:11:38

i'd like to parallelize jobs over 3 cores, queuing waiting jobs. right now, we're using pipeline-blocking to parallelize over 3 cores. looks like this: (pipeline-blocking 3 out-chan (map process) (to-chan jobs) true) i think this processes jobs 3 at a time--that's good. but i think that because we have numerous end-users who can kick off lists-of-jobs, we have potentially 3 x number-of-users-who-kick-off-jobs

zlrth00:11:59

it might be as simple as a top-level channel: (def queueing-channel (chan)) then

(put-all-elems-on-chan! queueing-channel jobs)
(pipeline-blocking 3 out-chan (map process) queueing-channel true)
but i'm having trouble finding the equivalent of put-all-elems-on-chan!

zlrth00:11:33

@hiredman sorry, i don't know which part of what i said doesn't make sense

hiredman00:11:52

I don't see what you are asking at all

hiredman00:11:00

what is the question?

zlrth00:11:54

i think that if i call (pipeline-blocking 3 out-chan (map process) (to-chan jobs) true) say 10 times, i'll get 30 jobs processing at the same time. i'd like a max of 3 jobs processing at once. i'm phrasing this in this way because i'm not sure if i'm right about pipeline-blocking

hiredman00:11:30

each call creates a new pipeline

hiredman00:11:48

if you only want one, find a way to only create your pipeline once

zlrth00:11:52

thanks. i now think that this is what i want:

(def queueing-channel (chan))
(put-all-elems-on-chan! queueing-channel jobs)
(pipeline-blocking 3 out-chan (map process) queueing-channel true)
i'm finding it hard to understand the async docs. my own fault, of course.

peter-kehl00:11:04

some-fn and every-pred - why don't they have same postfix (either -fn or pred)? Is there some convention/logic behind it, or only history, please?

noisesmith01:11:22

some-fn returns whatever your fn returned for the item, every-pred just returns true or false like a predicate

noisesmith01:11:40

Clojure 1.9.0
(ins)user=> ((some-fn :a :b) {:b "OK"})
"OK"
(ins)user=> ((every-pred :a :b) {:b "OK" :a "YES"})
true

peter-kehl01:11:22

Ah... Thank you @noisesmith.

jaawerth01:11:43

on a side note, it bugs me to irrational levels that some-fn and every-pred don't follow a common naming scheme, lol

andy.fingerhut01:11:56

As long as you realize you are not completely rational in how much it bugs you 🙂

😅 1
tballantyne05:11:19

hi folks, I'm setting up compojure, ring, java.jdbc and postgres. When I lein ring server I get String cannot be cast to clojure.lang.Ifn in core.clj during Thread.run

tballantyne05:11:35

Anyone see this before?

seancorfield05:11:44

@thomasb That suggests you have, essentially, (some-string-value ...)

seancorfield05:11:20

What does your Compojure routes form look like?

tballantyne05:11:57

hi @seancorfield, routes are here ^^^

seancorfield05:11:08

Line 11 ends with ) but the call to (views/home-page) should be inside that.

seancorfield05:11:58

And the ) at the end of line 12 is closing (defroutes ... -- it should be at the end of line 26.

tballantyne05:11:34

yeah, I see that now.

seancorfield05:11:36

I guess you're not using an editor that auto-balances parentheses based on indentation?

tballantyne05:11:53

pretty plain vim config

seancorfield05:11:15

You'll find it hard to edit a Lisp that way, I think.

seancorfield05:11:42

I'm not familiar with paredit/parinfer plugins for vim but that's what you'll want to look for.

seancorfield05:11:50

Let me know if those edits fix the problem and get your app running.

tballantyne05:11:36

yes, that was it. server is running. will look into a plugin for that. thanks for your help!

seancorfield05:11:31

You're welcome! I'm off for the evening. If you need help with java.jdbc, feel free to post in #sql and I'll answer when I can (I maintain that library).

tballantyne06:11:09

oh, nice. thanks. The db connection worked fine off the bat.

1
sb07:11:20

Hello, Somebody have an idea how write SD card from clojure?

sb07:11:44

Any idea, would be great!

andy.fingerhut08:11:51

Is this an SD card that is already formatted, and looks like a normal file system to the OS?

andy.fingerhut08:11:33

If so, then it should be the same as writing any other file that is on a disk or SSD drive.

👍 1
sb11:11:31

Ok, that isn’t easy .. if I think good, I read about. But I test it.. I can just tomorrow

sb11:11:50

Maybe true.

Eccentric J17:11:08

Hey there. Is there a best practice for the ordering of requirements in a namespace?

alexmiller17:11:25

I agree with 95% of that

Eccentric J17:11:45

Thanks a ton! What part do you disagree with?

alexmiller17:11:05

I don’t buy the argument for lists in :import

Eccentric J17:11:16

> Use lists (parentheses) for the top-level clauses such as (:require ...). Because the first element, the :require or :import keyword, is special.

Eccentric J17:11:21

I don’t understand that argument

Eccentric J17:11:37

Is the idea that anytime the first element is important you should use a list?

Audrius18:11:55

why to have a keyword with double semicolon? like in here https://clojure.org/reference/multimethods

hiredman18:11:40

the double semicolon is a short cut for namespacing a keyword

hiredman18:11:14

::foo will be read in qualified with the current namespace

hiredman18:11:31

Clojure 1.9.0
user=> :foo
:foo
user=> ::foo
:user/foo
user=> 

👍 1
hiredman18:11:14

user=> (require '[clojure.set :as set])
nil
user=> ::set/foo
:clojure.set/foo
user=> 

hiredman18:11:29

:: can also work with aliases created using require

Audrius18:11:09

second one is very confucing

Audrius18:11:50

is it like adding keyword to a namespace as you would add a var?

hiredman18:11:05

:: is often used in examples because it makes code shorter, but I don't like it in real code because it ties code to a particular namespace, if you have :: somewhere and then move that code to a different namespace it means something different

hiredman18:11:34

keyword namespaces are basically just a prefix

hiredman18:11:16

:: is actually the one place where var namespaces and keyword namespaces interact

Audrius18:11:14

I can see it still adds to that names pace because ::nonexitsting-ns/ok won't work

noisesmith18:11:25

that fails because it can't look up the alias

noisesmith18:11:40

:nonexisting-ns/Ok is fine

Eccentric J18:11:17

my-custom-ns=> (ns foo.bar)
nil
foo.bar=> *ns*
#object[clojure.lang.Namespace 0x1e71dad1 "foo.bar"]
foo.bar=> :my-keyword
:my-keyword
foo.bar=> ::my-keyword
:foo.bar/my-keyword
foo.bar=> (= ::my-keyword :foo.bar/my-keyword)
true
foo.bar=> (= ::my-keyword :my-keyword)
false

hiredman18:11:48

it doesn't add to the namespace

hiredman18:11:13

it just uses the var namespaces to figure out how to resolve the '::' shortcuts

johnj18:11:15

the word 'namespace' in clojure is confusing a first since it has two different meanings, one for keywords and the other for "group" of symbols

Audrius18:11:16

so single : creates "global" keyword? :thinking_face:

hiredman18:11:18

just imagine a keyword is a pair

Eccentric J18:11:44

Not quite. It’s just that :my-keyword will evaluate to the same value anywhere. But (= :foo.bar/my-keyword :other.ns/my-keyword) ;; => false

👍 1
hiredman18:11:51

a namespaced keyword is [ns name] a not namespaced kewyrod is [nil name]

🤓 1
hiredman18:11:14

:: is just shortcut syntax for filling in the first part of the pair

hiredman18:11:35

and the syntax happens to be based on the value of *ns* at read time

hiredman18:11:51

*ns* being the current var namespace

hiredman18:11:32

otherwise the keyword pairs don't interact with the var namespaces at all

Eccentric J18:11:06

A practical example might be distinguishing between similarly named keywords between separate use cases. For instance you may want to differentiate between :project.ns.db/connected and :project.http/connected and ::connected will either resolve to :project.ns.db/connected or :projected.http/connected depending on the value of *ns*.

👍 1
hiredman18:11:08

so :foo/bar is a valid keyword, and has the namespace part foo regardless of if the var namespace foo exists or not

1
👍 1
Eccentric J18:11:11

Another use case is in maps: {:project.ns.db/status :ok :project.http/status :ok)

Eccentric J18:11:47

and if you ran (ns project.ns.db) you could do (::status {:project.ns.db/status :ok :project.http/status :error}) ;; => :ok

Audrius18:11:32

and how about symbols? is it just something quoted? or is it more for it?

Eccentric J18:11:35

Symbols can be namespaced (quote +) ;; => + versus

`+ ;; => clojure.core/+

ccann19:11:35

is it possible to type-hint as a keyword?

noisesmith19:11:14

no, because keywords don't implement IMeta - but why would you need to hint something that's a literal in the code?

noisesmith19:11:25

if it's a symbol bound to a keyword, yes you can type hint symbols

ccann19:11:47

yeah I mean type hint a symbol to indicate its type is a keyword…

noisesmith19:11:16

keywords don't have separate instances - :foo and :foo will be the same object internally, so there's no place to put the metadata without putting it on all occurances

noisesmith19:11:30

right, ^clojure.lang.Keyword x

noisesmith19:11:21

oh oops, I misread your question, sounds like we've sorted it out anyway

alexmiller19:11:30

generally there is no good reason to do that

alexmiller19:11:51

type hints help with interop calls and your interaction with Keywords should pretty much always be via clojure code, not java interop

👍 1
ccann19:11:46

I was doing it as a form of documentation since it was 1 of several args to a function, some of which already had (useful) type hints

alexmiller19:11:56

It is a best practice to only add type hints when necessary and avoid type hints that have no value

ccann19:11:53

the value to me in this case was disambiguation, but yeah I follow

serg19:11:59

Hey everyone, I’m just trying ClojureScript and figwheel and wondering if this is ok:

(event/listen (dom/get-element "test")
              :blur
              (fn []
                (prn ":blur")))
This event handler doesn’t work when I run after production build js. But if I replace :blur to “blur” it does. So is it required? for advanced optimization?

Eccentric J21:11:01

Is there a recommended way to use spacemacs<=>cider<=>luminus repl?

noisesmith21:11:58

it's likely easier to launch your luminus app from a repl, but alternatively there's an nrepl function to start a repl from within your app if you add tools.nrepl as a dep

Jim Rootham21:11:22

I am trying to write a local library and I am getting.

Jim Rootham21:11:26

Could not locate satlib__init.class or satlib.clj on classpath.

Jim Rootham21:11:41

satlib is the library.

Jim Rootham21:11:25

I was thinking I failed RTFM but I looked at 'lein classpath' and it includes a jar with satlib .clj so I am suspecting a mystery (almost certainly caused by my doing something wrong)

Jim Rootham21:11:00

The code chunks

mfikes21:11:44

@jrootham You probably want (:use satlib.core) instead of (:use satlib)

Jim Rootham21:11:40

I tried that at one point. I have subsequently renamed core to satlib in src

Jim Rootham21:11:50

lemme look at something

mfikes21:11:25

When you require a library using :requre / :use etc, you are specifying the namespace you want to load. That namespace needs to be in a file following some mild rules. For satlib.core the file would be satlib/core.clj For satlib the file would be satlib.clj

mfikes21:11:31

@jrootham Perhaps you put satlib.core in a file named satlib.clj?

Jim Rootham21:11:53

As in the ns argument? Yup, but I just changed it and it didn't fix it.

Jim Rootham21:11:27

Should I migrate back to satlib.core?

mfikes22:11:56

@jrootham up to you—the thing you require just needs to line up with what is available on your classpath

Jim Rootham22:11:49

Is there a way to see what exactly is on the classpath, below the 'lein classpath' and 'jar tf" levels?

noisesmith22:11:06

like, opening the actual artifacts? your editor should be able to do that

mfikes22:11:17

I’m afk, but in a REPL you can use to try to poke around

noisesmith22:11:28

right, if I do (io/resource "some/file.clj") that printed result shows you the path where file.clj was actually found and if it was found (non-nil) you can open it and slurp or read or whatever

Jim Rootham22:11:38

But here the file is not available because it is not found.

mfikes22:11:13

If your JAR had everything in it under src/ for example, that would cause an issue

Jim Rootham22:11:23

I get "No such namespace:

Jim Rootham22:11:25

HOT.jar META-INF/MANIFEST.MF META-INF/maven/satlib/satlib/pom.xml META-INF/leiningen/satlib/satlib/project.clj project.clj META-INF/leiningen/satlib/satlib/README.md META-INF/ META-INF/maven/ META-INF/maven/satlib/ META-INF/maven/satlib/satlib/ META-INF/maven/satlib/satlib/pom.properties satlib/ satlib/satlib.clj

mfikes22:11:12

It is under satlib/satlib.clj

Jim Rootham22:11:16

That's the jar in the maven repository

Jim Rootham22:11:55

So, what silly thing have I done?

mfikes22:11:34

That would imply its namespace is satlib/satlib

Jim Rootham22:11:12

I have tried that, shall I try again and hope to get it right this time?

hiredman22:11:36

you need the correct thing in 3 places for it to work

hiredman22:11:58

1. the source file needs to have a namespace declared in it that matches its path

hiredman22:11:02

2. the path needs to be correct

hiredman22:11:22

3. the require or use needs the correct namespace

Jim Rootham22:11:30

So satlib/satlib in the project file and satlib.satlib in the use

hiredman22:11:09

so you can't just kind of go around changing things in place or the other and say it doesn't work, those 3 things don't need to be some specific value, but they need to match

hiredman22:11:43

the names in project.clj don't matter in terms of clojure namespaces

mfikes22:11:54

Following what @hiredman said, if in the JAR, it is named satlib/satlib.clj, in that file it has to have (ns satlib.satlib) and you have to require it using (:use satlib.satlib) (since you are using :use)

Jim Rootham22:11:04

The ns in satlib

Jim Rootham22:11:08

(ns satlib (:gen-class) (:require [clojure.set :refer [intersection difference union]]) (:require [clojure.string :as str]) )

mfikes22:11:26

Yeah, so in that case (ns satlib ...) doesn't match satlib/satlib.clj

hiredman22:11:39

the names in project.clj are generally related to maven artifact names, and don't need to match or be related in away to clojure namespaces that may be defined in files in that artifact

hiredman22:11:47

generally in clojure single segment namespaces like satlib are considered a bad idea (for reasons that have to do with how the compiler maps clojure code to jvm bytecode in classes)

hiredman22:11:22

the way namespaces map to paths on disk is more or less "." => "/" and "-" => "_"

Jim Rootham22:11:31

OK, that seems to work

hiredman22:11:07

so the namespace satlib would be expected to be found in satlib.clj, satlib.core would be expected to be in satlib/core.clj

hiredman22:11:26

sat-lib.core would be expected to be in sat_lib/core.clj

Jim Rootham22:11:54

Thank you both very much

Eccentric J22:11:58

When I run lein run with a luminus app running the snippet above and try to connect to the repl I get a ton of output and cider-connect hangs

noisesmith22:11:25

what about lein repl :connect with the port number?

Eccentric J23:11:01

web_1  |   [java.lang.Thread run Thread.java 748]]} Unhandled REPL handler exception processing message {:id 10d1b11a-937b-4849-a2d4-42c2206309ef, :op clone})

noisesmith23:11:18

I would avoid putting cider code in an app, also

noisesmith23:11:42

if you want cider features, start the app from inside cider maybe?

Eccentric J23:11:59

Like cider-jack-in?

hiredman23:11:22

it looks like you are using some kind of code reloading

hiredman23:11:42

which reloaded the nrepl code, which created new protocol definitions (with the same name as the old one)

Eccentric J23:11:49

Yeah I’m using the default luminus config, I only added the cider lines.

hiredman23:11:11

but the existing objects don't implement the new protocols

Eccentric J23:11:32

Oh wait, maybe this is my fault. I tried sending the code that defines the repl server to the repl 😛

Eccentric J23:11:16

I could maybe try cider-jack-in but I’m trying to run the luminus app from a docker container compsed with a postgres container for storage.

Eccentric J23:11:33

web_1  | 2018-11-16 23:04:03,434 [main] INFO  ts-api-explorer.core - #'ts-api-explorer.core/repl-server started
web_1  | 2018-11-16 23:04:26,768 [clojure-agent-send-off-pool-3] ERROR nrepl.misc - (#error {
web_1  |  :cause No implementation of method: :send of protocol: #'clojure.tools.nrepl.transport/Transport found for class: nrepl.transport.FnTransport
web_1  |  :via
web_1  |  [{:type java.lang.IllegalArgumentException
web_1  |    :message No implementation of method: :send of protocol: #'clojure.tools.nrepl.transport/Transport found for class: nrepl.transport.FnTransport
web_1  |    :at [clojure.core$_cache_protocol_fn invokeStatic core_deftype.clj 583]}]
web_1  |  :trace
web_1  |  [[clojure.core$_cache_protocol_fn invokeStatic core_deftype.clj 583]
web_1  |   [clojure.core$_cache_protocol_fn invoke core_deftype.clj 575]
web_1  |   [clojure.tools.nrepl.transport$eval20851$fn__20852$G__20842__20859 invoke transport.clj 16]
web_1  |   [clojure.tools.nrepl.middleware.session$register_session invokeStatic session.clj 144]
web_1  |   [clojure.tools.nrepl.middleware.session$register_session invoke session.clj 137]
web_1  |   [clojure.tools.nrepl.middleware.session$session$fn__21472 invoke session.clj 188]
web_1  |   [clojure.tools.nrepl.middleware$wrap_conj_descriptor$fn__21121 invoke middleware.clj 22]
web_1  |   [nrepl.server$handle_STAR_ invokeStatic server.clj 17]
web_1  |   [nrepl.server$handle_STAR_ invoke server.clj 14]
web_1  |   [nrepl.server$handle$fn__42728 invoke server.clj 26]
web_1  |   [clojure.core$binding_conveyor_fn$fn__5476 invoke core.clj 2022]
web_1  |   [clojure.lang.AFn call AFn.java 18]
web_1  |   [java.util.concurrent.FutureTask run FutureTask.java 266]
web_1  |   [java.util.concurrent.ThreadPoolExecutor runWorker ThreadPoolExecutor.java 1149]
web_1  |   [java.util.concurrent.ThreadPoolExecutor$Worker run ThreadPoolExecutor.java 624]
web_1  |   [java.lang.Thread run Thread.java 748]]} Unhandled REPL handler exception processing message {:id BB0741D0-3CEF-4F9C-AC1C-3D4630FF0A18, :op clone})

Eccentric J23:11:52

Still off. I’ll remove the cider lines and see if that makes it return back to normal

dpsutton23:11:47

i think there are two clashing versions of nrepl here. note clojure.tools.nrepl.transport and nrepl.transport.FnTransport. Lein has not had a release in a while and is still using deprecated contrib nrepl rather than the newer versions

dpsutton23:11:13

bbatsov wrote a plugin for lein to use cider/nrepl not tools/nrepl

Eccentric J23:11:29

Hmm should I try cider 0.17.0? Ah ok!

dpsutton23:11:47

that is most likely the easiest path

Eccentric J23:11:52

Removing cider restored it to normal. I can connect but when I do I see:

WARNING: CIDER requires cider-nrepl to be fully functional. Many things will not work without it!

dpsutton23:11:56

it's confusing but there are two "nrepl" things around. There's cider/cider-nrepl "0.19.0-SNAPSHOT" and nrepl "0.4.5" versus [org.clojure/tools.nrepl "0.2.13"]

Eccentric J23:11:05

Backing up a bit I think the way I organized this project with Docker is creating some issues.

Eccentric J23:11:26

ROOT
  |- webapp
  |  |- Dockerfile
  |  |- project.clj
  |- docker-compose.yml

Eccentric J23:11:54

Because now I can connect to cider but if I try to execute anything against the repl I see a no client repls in current session even though I’m looking at a cider repl pane right next to it.

Eccentric J23:11:37

Whoo got it working! Thanks a ton for the help

Ben Grabow23:11:41

If I have a mixed data structure like below, is there a way I can update-in the tail of a vector like this?

seancorfield23:11:53

@ben.grabow Have you looked at Specter?

Ben Grabow23:11:56

Looks like it will do just what i need. Thanks!