Fork me on GitHub
#clojure
<
2020-01-31
>
seancorfield00:01:49

Yeah, 1,024 https://github.com/clojure/clojure/blob/ee3553362de9bc3bfd18d4b0b3381e3483c2a34c/src/clj/clojure/core.clj#L7851 -- tap> returns true if it succeeds in enqueuing the data or false if it drops the data.

andy.fingerhut00:01:10

I guess motivation there may be "the first one/few are guaranteed to be tapped, but we didn't want the implementation to significantly slow down / blow up memory use if you stick it in a a hot code path"?

seancorfield00:01:21

Right, because it is primarily for dev/test work.

andy.fingerhut00:01:42

And I guess if you do want to stick it in a hot code path, and guarantee you get the ones you want to see (or at least increase the odds), you should be adding conditions that qualify the calls to tap> that cause it to be called only for values you really want to tap, not everything.

andy.fingerhut00:01:35

Most things called 'logging libraries' will back pressure if you log a lot, i.e. slow down your code, but are typically lossless, yes?

didibus07:01:22

That's a good question. I've always been told if you can't lose logs, don't use async logging. But my thought was always, because if the process crashes, or the disk fails, or the machine blows up, etc., you're going to lose the logs that are still in the queue.

didibus07:01:35

That said, now that I think about it, async logging needs a way to handle what to do if there's the logging can't keep up, seems like making it unbounded and possibly running out of memory and having your process crash is not a great default, so maybe they also just have some default bound and drop logs if that happens

lmergen09:01:43

I agree with this. Either you go "all the way" and immediately, synchronously send your logs to a log aggregator (i.e. "i really cannot lose these"), or you do asynchronous logging. Synchronous logging but relying on a server not to crash is the worst of both worlds, imho.

hiredman00:01:39

yes, I don't mean it is a logging library, I think there is some analogy to be made there. tap> doesn't actually tell you if a value is accepted or not, so you can do something like (while (not (tap> :whatever)))

andy.fingerhut00:01:03

Hm, yeah, the return value from tap> does make it straightforward to create that and call it lossless-tap> , if you really want it.

jaide01:01:57

;; In cmd.clj
(defn -main [& args] (println (slurp *in*)))
;; In shell
ack -g "js" | clj cmd.clj
Any reason come to mind why that's not working? Seems to run without errors but nothing is printed.

jaide01:01:27

Oh wait! I forgot to use -m I think.

jaide01:01:53

Yep. That did it :man-facepalming:

jaide01:01:08

Another question though, how would one read both a list of files piped from a unix program and use (read-line) to capture user input (like a y/n confirmation)?

andy.fingerhut01:01:20

I don't think you could use read-line for that. On Linux, you could do it by going to /dev/tty (or something close to that -- I haven't done it so could be off on the file name)

andy.fingerhut01:01:20

In such a program, read-line would be used for stdin, the unix piped input, and /dev/tty or whatever the correct name is, would be used only for the direct-to-terminal interaction with interactive user.

jaide01:01:11

That helps. Thank you!

dpsutton01:01:29

Maybe babashka would suit your problem space well

jaide01:01:44

(defn read-prompt
  "Reads the next line from stream that is the current value of /dev/tty ."
  []
  (let [tty (io/reader "/dev/tty")]
    (if (instance? clojure.lang.LineNumberingPushbackReader tty)
      (.readLine ^clojure.lang.LineNumberingPushbackReader tty)
      (.readLine tty))))
That worked creating a modified version of read-line. Though the type casting conditionals are probably unnecessary?

jaide01:01:51

babashka may have been useful but I just wanted to write a quick script to interactively prompt me to replace text in a bunch of files.

noisesmith01:01:12

"the current value of /dev/tty" reads oddly - /dev/tty is a pseudo-file which reads your inherited tty, which will be a different value when run in different contexts, but isn't mutable per-se

jaide01:01:36

The context also affects read-line in the same way right?

noisesmith01:01:57

right, or a child process, my only concern is that it reads strangely

noisesmith01:01:12

as if /dev/tty were something that gets mutated or reset

noisesmith01:01:57

you could as well say "the current value of /" (which is also a special file which takes on different values depending on how your process is started, thanks to chroot)

jaide01:01:45

https://gist.github.com/eccentric-j/650e92d2d958d51572e9a8a71ed847d2 the script worked and did the job I needed it to. So if that's the case what's the correct way to read user input when piping to a clojure program?

noisesmith01:01:32

it's posix specific (eg. no windows support), but yes that's correct

noisesmith01:01:01

@jayzawrotny I'm not saying your code is wrong, I'm just nitpicking the docstring

jaide01:01:47

Oh I misunderstood. My code could be wrong, it was a very haphazardly put together script. 😄

noisesmith01:01:44

and for all I know "current value of /dev/tty" is a commonly used term that I just haven't seen, it just looks weird to me

4
andy.fingerhut03:01:11

There is also a Java Console class that might be more portable between OS's, but that is only a wild guess on my part, without having ever tried it: https://stackoverflow.com/questions/8138411/masking-password-input-from-the-console-java

ag06:01:20

how can I use every-pred for functions that take more than one argument? e.g: how do make something like this work?

((every-pred (fn [a b]
               (prn "a in first" a)
               (prn "b in first" b)
               true)
             (fn [a b]
               (prn "a in second" a)
               (prn "b in second" b)
               true)) "a" "b")

didibus07:01:52

every-pred only works with functions of one argument

didibus07:01:10

But you could wrap them however you wanted

didibus07:01:54

So you could for example convert your preds of two args, into preds of one arg which takes a vector of two elements

didibus07:01:42

((every-pred (fn [[a b]]
               (prn "a in first" a)
               (prn "b in first" b)
               true)
             (fn [[a b]]
               (prn "a in second" a)
               (prn "b in second" b)
               true)) ["a" "b"])

ag07:01:27

ah, I see… thank you

ag07:01:04

and also thank you for taking your time writing the answer in clojureverse as well

didibus07:01:54

No problem

didibus07:01:11

I just updated it actually, to show a more general approach of the above, could be useful

didibus07:01:22

That's a good question. I've always been told if you can't lose logs, don't use async logging. But my thought was always, because if the process crashes, or the disk fails, or the machine blows up, etc., you're going to lose the logs that are still in the queue.

didibus07:01:35

That said, now that I think about it, async logging needs a way to handle what to do if there's the logging can't keep up, seems like making it unbounded and possibly running out of memory and having your process crash is not a great default, so maybe they also just have some default bound and drop logs if that happens

borkdude09:01:53

@jayzawrotny fwiw I find projectile-replace (in emacs) useful to search and replace things in all files of a project

jaide16:01:11

Oh that's helpful! In this case though I wanted to replace every instance of "pluck" with "path" found in paths like subapp/**/use_cases/*.js. Not all instances either, only in request blocks so I wanted confirm functionality as well.

zilti16:01:26

I'm trying to use refs, but I am failing... I assumed that a dosync block is like a transaction, and that only one dosync block operating on a given ref can be run at any single time?

vemv16:01:13

What specific issue are you having? > only one dosync block operating on a given ref can be run at any single time? Not exactly... I'd recommend picking up a clj book, the topic is quite fun to learn and you might get an overly condensed take if asking via Slack :)

zilti16:01:40

Thing is that so far I have only used Atoms when multithreading, they were "good enough". Now I need something blocking... Neither refs nor agents provide that apparently though.

vemv16:01:48

clojure.core/locking actually blocks but knowing what you're doing is a must there Other blocking or pseudo-blocking alternatives include BlockingQueues, core.async BQueues are explained in Clojure Applied

zilti17:01:27

What I am doing is I am starting a pool with three headless browsers. Thing is I want that pool initialized on first access, so chances are near 100% that more than one thread will try to access it while it is already initializing.

zilti17:01:10

So basically, I need a construct that blocks on deref until an ongoing update/transaction is finished

vemv17:01:52

future / delay / promise?

zilti17:01:41

I need to be able to alter the content though.

zilti17:01:55

I guess I could use locking together with refs

markmarkmark17:01:14

you could have a promise that is available to the threads that is only delivered once the pool is finished initializing

vemv17:01:36

Could be locking (preferably without refs), could be refs integrated with agents (they have a special relationship)... Obviously the better you know clj's offering, the more alternatives you'll be able to design

hiredman17:01:12

sounds like you just want a delay

Alex Miller (Clojure team)18:01:20

You want a CyclicBarrier

👀 4
Alex Miller (Clojure team)18:01:00

It’s in the jdk, it’s the ideal construct for this. Or a CountdownLatch

Alex Miller (Clojure team)18:01:25

Phasers are fancier version, probably more than you need

didibus18:01:21

Ya, it really sounds like you just want delay

didibus18:01:03

(def d (delay (Thread/sleep 5000) 100))
(time (mapv deref [(future @d) (future @d) (future @d)]))
"Elapsed time: 5083.495667 msecs"
[100 100 100]

didibus18:01:25

Delay blocks on deref, and only initializes its content on first access. Now, if you're going to make updates to a mutable thing stored inside the delay later on, you need another strategy to synchronize those.

didibus18:01:36

promise can be used the same if you want another thread to do the initializing

noisesmith18:01:23

@U2APCNHCN > I need something blocking... Neither refs nor agents provide that apparently though. only one action can be applied to a given agent at a time, unlike refs and atoms they never retry

didibus19:01:07

But agents don't block on read

didibus19:01:23

Which sounds like what they are asking

jumpnbrownweasel19:01:22

@U2APCNHCN Is it true that each browser/driver can only be used by one thread at a time, yet you want to use more than one browser concurrently? If so, I think you just want an exclusive lock per browser while initializing or using that browser, rather than a lock on a pool containing all three browsers. clojure.core/locking gives you an exclusive locking mechanism.

zilti21:01:52

Yea block on read would be kind of important. locking was what I needed though

nickmbailey18:01:57

did transit-js get superceded by EDN?

macrobartfast19:01:08

I am trying to get rolling with using deps.edn as opposed to project.clj and when I run clj at the prompt in a directory it never seems to be aware of the deps.edn in that directory... help.

ghadi19:01:11

paste your deps.edn @macrobartfast

ghadi19:01:54

also you can run clj -Sverbose and it should show three deps.edn's under config_paths

ghadi19:01:01

(system, home & local dir)

Alex Miller (Clojure team)19:01:59

Maybe syntax in the deps.edn?

macrobartfast20:01:14

{:paths ["src"]
 :deps
 {io.pedestal/pedestal.service {:mvn/version "0.5.7"}
  io.pedestal/pedestal.jetty {:mvn/version "0.5.7"}
  com.layerware/hugsql {:mvn/version "0.4.9"}
  org.clojure/tools.namespace {:mvn/version "0.2.11"}
  org.clojure/tools.logging {:mvn/version "0.3.1"}
  ch.qos.logback/logback-classic {:mvn/version "1.1.3"}}
 :mvn/repos
 {"central" {:url ""}
  "clojars" {:url ""}}}

macrobartfast20:01:20

does that look ok?

andy.fingerhut20:01:05

When I create a new empty directory, create a deps.edn file that I copy and paste that text into, save it, and run clj in that directory, it downloads many JAR files described in there that were not already in my $HOME/.m2 directory, so it seems to be reading it and using its contents.

andy.fingerhut20:01:27

What behavior do you observe?

λustin f(n)20:01:42

Are there any tools or techniques to figure out what a repl is spending its time on while starting up? My project's startup time is getting out of hand, to the point of timing out lein repl sometimes.

borkdude20:01:37

I have trouble typing your name because it starts with a lambda, but yes, you can try to set (bind) clojure.core/*loading-verbosely* to true

12
andy.fingerhut20:01:56

@ghadi The command clj -Sverbose shows me the same three deps.edn files whether there is a deps.edn in my current directory, or not. Is that normal, and if so, how would it help here?

Alex Miller (Clojure team)20:01:29

It’s telling you where it’s looking

andy.fingerhut20:01:56

I mean, I guess it would help if there was a different home-grown bash script named clj in your path.

andy.fingerhut20:01:08

which I have had, way back when

ghadi20:01:31

ah, never mind

ghadi20:01:03

@macrobartfast are you sure that you're in the same directory as the deps.edn?

ghadi20:01:39

the :mvn/repos part isn't strictly necessary, but looks fine

macrobartfast20:01:32

I'm in the same directory

macrobartfast20:01:58

clj -Sverbose produces Exception in thread "main" .FileNotFoundException: -Sverbose (No such file or directory)

borkdude20:01:25

what does which clj print?

borkdude20:01:44

maybe you have some alias or some other program which took that name

macrobartfast20:01:56

/usr/local/bin/clj

borkdude20:01:36

hm, looks fine.. what does clj --help print? (the first few lines)

macrobartfast20:01:03

Exception in thread "main" .FileNotFoundException: -help (No such file or directory)

borkdude20:01:13

(sorry, double hyphens)

andy.fingerhut20:01:15

I suspect if you type cat /usr/local/bin/clj you will see something significantly different than the clojure/clj version of the bash script, which looks like this:

#!/usr/bin/env bash

if type -p rlwrap >/dev/null 2>&1; then
  exec rlwrap -r -q '\"' -b "(){}[],^%#@\";:'" clojure "$@"
else
  echo "Please install rlwrap for command editing or use \"clojure\" instead."
  exit 1
fi

macrobartfast20:01:41

Usage: java -cp clojure.jar clojure.main [init-opt*] [main-opt] [arg*]

  With no options or args, runs an interactive Read-Eval-Print Loop

  init options:
    -i, --init path     Load a file or resource
    -e, --eval string   Evaluate expressions in string; print non-nil values

  main options:
    -m, --main ns-name  Call the -main function from a namespace with args
    -r, --repl          Run a repl
    path                Run a script from a file or resource
    -                   Run a script from standard input
    -h, -?, --help      Print this help message and exit

  operation:

    - Establishes thread-local bindings for commonly set!-able vars
    - Enters the user namespace
    - Binds *command-line-args* to a seq of strings containing command line
      args that appear after any main option
    - Runs all init options in order
    - Calls a -main function or runs a repl or script if requested

  The init options may be repeated and mixed freely, but must appear before
  any main option. The appearance of any eval option before running a repl
  suppresses the usual repl greeting message: "Clojure ~(clojure-version)".

  Paths may be absolute or relative in the filesystem or relative to
  classpath. Classpath-relative paths have prefix of @ or @/

borkdude20:01:07

that seems like a very ancient thing?

andy.fingerhut20:01:12

There might be some other package you installed on that system besides the Clojure CLI tools before (or after), that is conflicting in the clj command name.

andy.fingerhut20:01:43

Yeah, or an older version

borkdude20:01:54

@macrobartfast wipe clj etc from your system and re-install the newest version. the newest version will also print the version when you type --help.

macrobartfast20:01:18

ha, ok cat /usr/local/bin/clj produces

#!/bin/sh
# Parts of this file come from:
# 

BREAK_CHARS="\(\){}[],^%$#@\"\";:''|\\"
CLOJURE_DIR=/usr/local/lib/clojure
CLOJURE_JAR=$CLOJURE_DIR/clojure.jar
CLASSPATH="$CLOJURE_DIR/*:$CLOJURE_JAR"

while [ $# -gt 0 ]
do
    case "$1" in
    -cp|-classpath)
            CLASSPATH="$CLASSPATH:$2"
    shift ; shift
    ;;
-e) tmpfile="/tmp/`basename $0`.$$.tmp"
    echo "$2" > "$tmpfile"
    shift ; shift
    set "$tmpfile" "$@"
    break # forces any -cp to be before any -e
    ;;
*)  break
    ;;
esac
done

if [ $# -eq 0 ]
then
  exec rlwrap --remember -c -b $BREAK_CHARS \
          java -cp $CLASSPATH clojure.main
else
  exec java -cp $CLASSPATH clojure.main $1 -- "$@"
fi

borkdude20:01:19

or just brew upgrade, depending on how you installed it

macrobartfast20:01:05

more tutorials and demos are using deps.edn, so that's how this came about for me. a good thing.

noisesmith20:01:06

yeah, that's not what we mean by clj, that's just a convenience script to run clojure.jar with no extra deps back when clojure could be run from a single jar

borkdude20:01:06

ok, problem solved 🙂

macrobartfast20:01:18

indeed, awesome!! very grateful.

isak22:01:07

Is there something in clojure.core similar to some->, except as a higher order function?

Alex Miller (Clojure team)22:01:15

can you explain what that would look like?

Alex Miller (Clojure team)22:01:31

are you looking for something like fnil for nil patching?

isak22:01:12

yea similar, but i guess for nil punning other functions. Example:

(defn something [f] (fn [x] (when (some? x) (f x))))
((something #(Long/valueOf %)) nil) ;;=> nil

isak22:01:14

hm that is similar, but would convert nil | String to nil | String | boolean

noisesmith22:01:19

@isak it's not the same as, but reminds me of, fnil

isak22:01:50

@noisesmith yea, I like that one, and it is what made me wonder

noisesmith22:01:12

and it's vararg!

(cmd)user=> ((fnil + 1 2) nil 1)
2
(cmd)user=> ((fnil + 1 2) 2 nil)
4

borkdude22:01:33

fnil isn't vararg, it takes between 2 and 4 params

borkdude22:01:23

the resulting function of (fnil + 1 2) is vararg of course, not sure if you meant that

noisesmith22:01:34

I was confused, clearly

noisesmith22:01:58

so you can only define defaults for the first three args

seancorfield22:01:30

@isak Your something function is basically (some-> x f) so (some-> v Long/valueOf) would match your example there (and you don't even need to wrap it in an anon fn).

isak22:01:59

@seancorfield I was passing it to a map, so needed a function, but I guess I could just do what you suggested, except with # and % (function shorthand syntax)

seancorfield22:01:21

#(some-> % f) you mean?

stand23:01:04

Is it possible to have more than one implementation of datafy for a given object type? I've only seen examples of one but what if my map or object can be datafied differently under different circumstances?

isak23:01:12

yea, exactly

seancorfield23:01:23

@stand That's the same issue with any protocol: once you extend it to a given type, it is "global" for your application and you can't have a different implementation for that type.

seancorfield23:01:59

What you can do with some protocols, is provide the behavior via metadata on the object instead of a static protocol extension.

seancorfield23:01:33

That only works with objects that support Clojure metadata tho' -- so you can't use it with raw Java objects, for example.

seancorfield23:01:53

(but you said "map" so I figured "Clojure hash map" which can have metadata)

stand23:01:15

Hmmm... so in Stu's intro talk how did he represent a vector as both a table and a graph? What that done via metadata?

seancorfield23:01:07

The viewer (in REBL -- which is what I think you're talking about) operates on data. You can select different viewers for a given data structure.

seancorfield23:01:30

That's not done via datafy -- it's a set of views that are available after you've produced data.

stand23:01:31

I see, so that's only a REBL thing then?

noisesmith23:01:10

would it make sense to make a "container" type that has custom behaviors for its contents? that way you could potentially display foo in two different ways via (container-a foo) and (container-b foo)

seancorfield23:01:18

(datafy some-vector) just produces the vector since it's already pure data. REBL can treat a vector as a table, raw EDN, a graph, etc.

👍 4
seancorfield23:01:56

@noisesmith Sure, you can create wrappers that have different datafy implementations for their contents -- but you can't easily use such wrappers in place of the underlying data in your code, and you'd have to be careful not to have your wrapper discarded by some intermediate Clojure operation.

lilactown23:01:07

to reiterate and add to what sean is saying, REBL does the work of dispatching on the type of the data it receives and determining which ways it can be visualized

lilactown23:01:41

so if the result of (datafy foo) is a vector, you can view it as a table and a graph

lilactown23:01:58

if the result is a map, other viewers can be used

lilactown23:01:16

they’ve talked about adding custom viewers but it hasn’t materialized yet

lilactown23:01:49

the way datafy works with REBL is that by extending the necessary protocols to datafy whatever object you’re looking at, you can then view it in REBL in a nice way

seancorfield23:01:53

For example, next.jdbc produces result sets whose rows are Datafiable such that if you datafy a row, you get a Navigable version, that will lazily navigate across FK relationships from columns in that row to other tables/rows in the database.

chrisulloa23:01:47

> I’ve only seen examples of one but what if my map or object can be datafied differently under different circumstances? Maybe the datafy function can inspect the object it’s datafying and then give different outputs? If that’s your goal.