if i have a defmacro inside a defmacro what’s the correct way to access the inner macro’s &env?

is it ~'&env?

yes, it should shadow the other

but you might find things much nicer if you can avoid nested defmacro

currentoor - wait, to access the inner &env, use enough ~ to get out of the inner macros but not enough to get out of the surrounding macro's

you can write a function that takes the macro form and returns the processed / fixed form, it might help restore some sanity

@noisesmith looks like ~'&env is working

sounds right - I had to be more general because I can't assume how many layers of are in play - I write macros that have no , I write macros that have multiple levels of `...

i agree nested macros should be avoided but i have a particular use case where the performance improvements and connivence justify it, IMHO

sure, but even then you can just make a defn that takes a list of symbols as an arg, and returns a new list of symbols that happens to contain defmacro

I'm using the weechat slack interface, i just realized most of what I wrote above looks weird because of how slack uses `

lol no worries

not sure what you mean by > returns a new list of symbols that happens to contain defmacro

a macro is a function (handled specially by the compiler) that happens to return a list of symbols for the compiler to eval

which means that you can often fix the problem of a macro being too complex by writing a function that takes a list of symbols (the args to the macro) and returns a list of symbols (the expansion)

thanks but i want to turn (div {:foo "bar"}) into (js/React.createElement #js {:foo "bar"}) in cljs and (div-element {:foo "bar"}) in clj

for all the html elements

i’m not sure that would help here

just curious, I have some code writing a huge collection to a file, the collection being generated through vanilla code of mine involving some recursion. Running the code, I observe all of my cores close to 100% utilized. It's always nice to see hardware fully utilized, but how can this actually be? Nothing special in how I write to the file either

(with-open [file (  "output.txt"  :append false)]
  (doall (map #(.write file (str %)) result)))

your core is pegged rrying to run the gc to recover memory would be my guess

don't use map+doall use doseq

mmmm good guess, I should profile it

Or run!

Just a thought ― would GC really utilize 12 cores though?

A stretch at a theory: If result is lazily computed, perhaps using pmap, it could crank up all the cores

@matan the jvm gc is concurrent, it can use all the cores you let it use

yes, result is lazily computed, as lazy is the clojure default, but I'm not using any pmap in my own code

and regardless of whether the gc is the issue here, use run! because nobody needs those objects that it is putting in your heap otherwise

and for other lazy things that you don't need to keep in memory check out dorun

I dunno, the use of doall+map like that would lead to me to believe there could be all kinds issues elsewhere

okay, re-reading about run! and dorun

either of those will work. doall retains the head of the sequence

oh right @ghadi

Think of doall as an accordion stretching out a seq, and think of run/dorun as lighting a fuse on one end of the seq and burning through to the end

With run! really doing well on directly-reducible collections

Indeed using run! my cpu utilization is now dramatically reduced (pun not intended :))

Though I still have it occupying ~50% of the total cores' time

What would be your advice? or is it essentially inevitable having so much GC when looping in a tight loop generating data in clojure? I guess If I were to also incorporate something like pmap for generating the data, it would be at rough competition with the GC activity.

your loop isn't tight

because results is lazy, your loop body is effectively computing each part of it

which is wrong because? what am I missing?

the data has to be computed I mean

sure, you want to compute something, but calling that loop a tight loop when it is doing some arbitrary computation in the middle of it which you haven't described is odd

I'm not sure how you mean that, but I tend to think it is just a typical artefact of clojure's immutability paradigm ― lots of objects being allocated when using an idiomatic clojure coding style centered around immutable data. https://purelyfunctional.tv/article/java-generational-garbage-collection/

I mean, because results is lazy, whatever computation (who knows what, you haven't said what it is) that goes in to building each step of it, is now happening in the middle of your loop, so it cannot really be considered "tight" (a tight loop meaning a loop that does very little each iteration and executes many times)

because of the laziness, there is not enough context in the loop to determine what is going on when the loop loops

so it is difficult for a person you just share the loop with to tell you what is happening

Yes of course, drop the "tight loop" term then; just a loop. There's a lot of CPU intensive code involved in generating the data, and I assume it is the general clojure case of creating many objects when deriving data from data in very idiomatic clojure code.

it could be, but I would be hesitant to assume that

I think sharing the specific code would just be tedious for anyone out here

sure

my assumption, given the doall+map code is you are not very familiar with clojure and there could be any number of problematic idioms lurking in there

@matan reflection generates garbage too

Anyone using Circle CI 2.0 for their clojure projects… are you building your own Docker images with Oracle JDK? Testing only with the available OpenJDK images? Using VM executors? ?

do you mean for the actual build/test process or for the result?

i use https://hub.docker.com/_/clojure/ with the lein tag

for building

for the result i use openjdk:9-jre

and copy the uberjars into it via Dockerfile

I meant for the testing. For instance, projects that build and test against Oracle JDK 7,8, and 9 and OpenJDK 7,8, and 9 in order to confirm no JDK specific issues. The official clojure images on dockerhub are OpenJDK based, only.

sounds like you might need to make your own images in that case

@jgh so it seems.

I was messing around with storing deps as the first form of the clojure source file itself.

'{:deps {cheshire {:mvn/version "5.8.0"}}}
(ns foo.bar (:require [cheshire.core :as json]))
(println (json/parse-string "{\"a\": 1}"))

#!/bin/sh
FILE=$!
clj -Sdeps "$(clj --eval '(fnext (read-string (slurp "'"$FILE"'")))')" -i $FILE

@yogidevbear Sounds like you want something like a flood-fill algorithm?

(defn flood-fill [matrix starting-pos replacement]
  (let [target (get-in matrix starting-pos)
        paint (fn [m pos]
                (cond-> m
                  (= (get-in m pos) target) (assoc-in pos replacement)))
        neighbours (fn [m pos]
                     (when (= (get-in m pos) target)
                       (for [dir [[0 1] [0 -1] [1 0] [-1 0]]]
                         (mapv + pos dir))))]
    (loop [matrix matrix stack [starting-pos]]
      (if (and (seq stack))
        (let [pos (peek stack)]
          (recur (paint matrix pos) (into (pop stack) (neighbours matrix pos))))
        matrix))))

@foxlog clj -Sdeps "{:deps {hello {:local/root \"/tmp/hello-world\"}}}" -m hello something like that

@foxlog clj and deps.edn are designed to primarily work in a “project” context where you have the deps.edn in the current directory. The example from @delaguardo treats it like a local dependency instead.

If your question is really packaging and deployment, that’s a bigger topic

it would be nice if you could link to subsections in a guide on http://clojure.org.

E.g. here I would like to link to a section: https://clojureverse.org/t/how-to-run-clj-from-an-arbitrary-directory/1665/2?u=borkdude

@borkdude you can: https://clojure.org/guides/deps_and_cli#extra_paths

click the links in the table of contents

oh, didn’t notice that, thanks

All headers get an anchor by default. Ideally they would have something that showed up in the gutter on mouse over to easily grab those

Gah, I am getting a weird error

zippers?

zip-visit is convenient for pre/postwalking https://github.com/akhudek/zip-visit

or specter might do it for you

I'm building an webservice with compojure-api 1.1.12 and I'm having trouble getting the swagger integration to show my responses. This is what I have

(PUT "/replay/:message-id" []
      :header-params [authorization :- String]
      :path-params [message-id :- String]
      :summary "Replay stuff"
      :swagger {:data {:description "Replays stuff"
                       :responses {200 {:description "reply stuff" :schema s/Str}}}}
      (ok "true"))

nevermind i figured it out

what is the channel to ask/learn about clojure deps/cli stuff?

#tools-deps

I have a question on conch (github Raynes/conch)…couldn’t find a channel for it so I’ll throw it out here and see if anybody has an idea. I’d like to spawn a process and while the process is running, print out the process output line-by-line. It seems to me (at least from my testing at the repl) that conch always completes execution before you get a chance to mess with the process out/err. If anybody has another clojure lib for dealing with shell execution and stream handling like this that would also be great. I know I can hand roll this myself by going directly against java process etc, but was hoping not to. Oh and from what I understand clojure.java.shell seems to have the same limitation

I'd hand roll it tbh

Grab the stdout inputstream and read it on a background thread while pumping into a channel or wherever

Usually these libs give access to the streams

right, yeah, I’ve done this in groovy more times than I’d like to admit so not a blocker, was just hoping for an easy/ier out

@mbjarland I found ProcessBuilder to generat a Process was much easier than using conch

Can someone point me at relevant documentation on whether I can assume that

(= Float/POSITIVE_INFINITY 
         (- (*
             Float/POSITIVE_INFINITY
             Float/POSITIVE_INFINITY)
            23))

@mbjarland from the Process you can get InputStream (the stdout) ErrorStream (the stderr) and OutputStream (the stdin)

@qqq the floating point spec

Is there any way to filter exceptions during a test run? I'm using pedestal, which includes the config for my entire interceptor chain, and I'm getting 8000-line stack traces.

@eraserhd I'm sure there's a way - with ring you can use a middleware that swallows all exceptions inside it, maybe someone in #pedastel can reveal the interceptor version of that

Oh, that didn't occur to me, I'll go ask there.

I've a question re: Using Transients inside reducers/fold - particularly inside combinef and reducef - I am trying to do a apply merge-with over transient coll inside combinef - Is there a way to do merge on transients? Thank you.

@bajracher calling persistent! on a transient is cheap - it's literally flipping a bit, and if you are merging you are likely done with the transient

which makes me wonder, is calling transient also cheap? I forget

yes, it’s constant time

awesome - hoping it's a low constant :P

`public TransientHashMap asTransient() { return new TransientHashMap(this); } ... ok looks relatively cheap https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentHashMap.java#L291

@noisesmith - mainly my motivation for using transient was also to be space efficient - not sure if that makes sense - I will be mutating the intermediate results quite a bit - its my impression that the persistent version will actually end up occupying more space as a result of immutability

@bajrachar all persistent! does is flip a bit, they use the same internal storage, in fact they are the same object except for some extra data storage the transient uses

@bajrachar the usage of transient isn't to reduce memory consumption - all hash-maps and vectors use the same internal tree structure following the same hashing and expansion rules when transient as when persistent - it's to reduce production of garbage and allow reassignment inside the original object instead of cloning leaves

the difference is that instead of making a new tree that replaces a leaf, you edit a leaf - same memory usage in resulting structure, less short term garbage creation

if your priority is to use less memory, just use a mutable collection (but use it smartly)

that’s not how transients work

@alexmiller oh? apologies

I'd appreciate your explanation

no time atm

or I guess I’d say that shortchanges the details

OK

@noisesmith @alexmiller - Thank you for taking some time to answer my question

I am basically trying to generate a index - and keep things in memory to do the intermediate processing

bajrachar for reference this is the official doc, I think it contains the important points here

https://clojure.org/reference/transients

take a look at this for a very well-written explanation of how transients are implemented: http://hypirion.com/musings/understanding-clojure-transients

I've used in-memory db - like lmdb - it seems a bit slow on writes -

so was trying to accomplish with using just clojure data structures - it seems to take a lot of memory

I may have missed the mark above, but everything I'm reading here still makes me think a transient would use the same memory in the end that the persistent version did, though I'd be interested in hearing why that's not so if it isn't

@noisesmith - my understanding with immutability was that it basically retained all intermediate changes - thus taking more space than say a local transient would

the intermediates go out of scope and get garbage collected

the difference in a transient is that you don't generate that garbage in the first place (and don't pay for fresh allocation) - you just mutate

the retained memory usage is the same

less churn

ok - I see - so I guess in that case there is really nothing for me to gain by using transient - 🙂

they are faster though

true

Well - I have things working by using an in-memory db - particularly - lmdb

however was looking to see if I could just use data structures instead

they just seem to take up more space

yeah, it's a space / speed optimization baked into the design

Thanks @noisesmith - for clarifying this

@bajrachar as I read more, there’s a “compress” of the transient that happens when you call persistent! but it’s O(1) and I think the compressing is limited

it would be interesting to get input from someone with better understanding of the code though

Also apparently you can't make a List transient?

I guess that makes sense

right, there'd be no benefit

taking your original question more literally, (conj! t0 (persistent! t1)) is effectively merge

just don't use t1 again

(and just like merge, use the return value and don't use the original again, etc.)

ok - I will give it a try - thanks

also - found an example in the doc for assoc! - https://clojuredocs.org/clojure.core/assoc!

example for merge

that doesn't do anything conj! doesn't do already, and I doubt it would do it better :p

and - it's not necessarily merging two transients in the example -

right, but that's just a mutatis mutandis issue

I asked this earlier but I didnt get any advice. Does anyone know how I can generate some POJOs to use with a library that calls jackson and determines how to build a response through reflection

My thought was to use (:gen-class), but I want to have a macro for this and I dont think gen-class is usable in that way.

@emccue the easiest thing is likely deftype

if all you need is fields that just works, if you need gettors/settors you can use defprotocol to make those, then deftype to implement

When is metadata evaluated when constructing a def?

depends which metadata you’re talking about :)

:)

(def ^{:hi 1} a 2}

var meta is evaluated, function signature metadata is not

I guess I'm curious about order. Is the var constructed first, and then the user-supplied metadata is evaluated and added to it? Or does the metadata get evaluated first, then the var is created, then it's attached?

and then there's the value attached to the var - that's three things

I know the var is created in the ns before the value is realized and attached to it

@noisesmith you're a vi guy right?

https://github.com/bhauman/rebel-readline/issues/127

could use some eyes on the above issue

@bhauman I'll check it out

thanks!

@bhauman this would be in command mode only - the way a "leader" works is that (in this example) "\" would put us into a special dispatch state, then "e" would edit, "a" would apropos etc.

the thing is to implement a leader you basically need a "mode" state that detects that a previous keypress introduced a leader, and also turns it off again when the command under that leader is invoked

so you might want to wait for a PR that implements a leader in rebl-readline

leader is basically just another keymap

it operates the same way Esc does in emacs

(in that it doesn't have to be toggled at the same time to act as a modifier)

@chris ahh, where escape prefix can replace meta, yes

@noisesmith I’ve already made these changes in my local rebel-readline and the bindings work as expected, at least in normal mode 🙂

@schmee oh, so it's easier than I thought? cool

I think leader preference is pretty particular, I've seen "," and " " as leaders for example

anyone know if leader exists in gnu-readline? I'm wondering if there's a spot for in it eg .inputrc

because yeah, I've always used space as leader

agreeing on keybindings for a thing like this is going to be impossible, config is the only way 🙂

I don't think readline even understands the concept of a leader?

except the hard-coded emacs-compat of escape

yeah the leader is just another keystroke here

in the readline

Is #datomic accessible from here?

Does anybody have experience with the google app engine standard environ and clojure after they released Java 1.8 and removed the whole whitelist debacle?