Fork me on GitHub
#clojure
<
2020-07-18
>
seancorfield02:07:18

The existing from-java function has not changed (although I found and fixed a small bug with from-java-shallow while working on this). This is therefore backward compatible, by using a new name for the correct behavior.

parens 4
❤️ 1
jsyrjala09:07:49

Are there any methods or tools for detecting if a lazy seq gets realized at wrong time. For example I have a large finite lazy seq which I’d like to realize at specific point in the program. The lazy seq goes through complex processing where items are modified etc. And then there is some code like (count lazy-seq) which realizes the lazy seq by mistake before the right place.

jsyrjala09:07:49

Are there any methods or tools for detecting if a lazy seq gets realized at wrong time. For example I have a large finite lazy seq which I’d like to realize at specific point in the program. The lazy seq goes through complex processing where items are modified etc. And then there is some code like (count lazy-seq) which realizes the lazy seq by mistake before the right place.

jsyrjala09:07:27

Possibly lazy seq could contain something that throws exception when realized, unless it is gets realized after certain point in program?

noisesmith14:07:16

it honestly sounds like you are misusing lazy-seqs here

noisesmith14:07:57

if it depends on side effects, and needs to be realized at a specific time (no sooner), it's better to use a procedural / side effecting low level tool, because that's what your domain actually requires

noisesmith14:07:33

of course a redesign that doesn't make your domain side effects (or minimizes the part of your code using that domain) is often an improvement, but that's a bigger design problem

jsyrjala08:07:26

In my case the side effects are HTTP GET requests that fetch data which will be part of the final the datastructure.

jsyrjala08:07:05

And the problem is running out of memory if the lazy seq is realized fully at the wrong point.

Ivan Fedorov09:07:16

Is there a special term when in a tree graph you transfer [node->leaf] edges from children to the root?

noisesmith14:07:39

its a kind of flattening

noisesmith14:07:47

I mean, "flatten" as we usually know it is a simpler subset of this operation for directed, non-looping graphs which disallow a node having two parents

Ivan Fedorov14:07:20

thanks! if I called it lift – do I miss?

noisesmith14:07:00

that also works I think - it might be confused with the concept from monadic / functor programming in ML family languages though(?)

noisesmith14:07:38

in haskell or OCaml etc. you can use an operator to "lift" a function on x to a new one on (functor x)

noisesmith14:07:59

eg. map lifts a function to operate on the seq monad in clojure

Ivan Fedorov21:07:49

got you, thanks!

ertugrulcetin17:07:12

Hi guys, I get clojure.lang.Compiler$CompilerException Syntax error compiling at when using lein test , but when I send (run-tests) to the REPL all works: {:test 8, :pass 16, :fail 0, :error 0, :type :summary} , any ideas?

noisesmith17:07:22

(run-tests) doesn't find and compile test files, it only runs the ones that are already loaded

noisesmith17:07:44

try (re)loading the test namespace that lein errored on

noisesmith17:07:06

(require 'some.ns :reload) (or whatever the equivalent is for your tool integration of choice)

ertugrulcetin17:07:13

there is just one test namespace, let me try

noisesmith17:07:56

what's important, to be pedantic, is how many test files there are, for lein test, because it finds and loads test files, (including ones the repl would never load because eg. they are shadowed)

noisesmith17:07:18

it's file tree oriented where clojure is classspaath oriented

ertugrulcetin17:07:47

I need that lein test command works without problem

noisesmith17:07:15

right, of course, I'm just trying to be more clear about reasons lein test would fail and a repl wouldn't

noisesmith17:07:31

also, which file does it fail to compile, the test file?

noisesmith17:07:45

if so, the require with :reload should replicate the error in the repl

noisesmith17:07:11

or (load-file "test/path/to/file.clj") to even more directly do what lein test does

noisesmith17:07:21

which is explicitly load files by path, not symbolically

ertugrulcetin17:07:31

(load-file "test/path/to/file.clj") returns nil

noisesmith17:07:52

what is the syntax error you get from lein test, can you show the full output?

ghadi17:07:12

you need to look at the entire error and analyze it @ertucetin

ghadi17:07:33

Syntax error compilng... which file?

ghadi17:07:39

that's the one that is causing the problem

ertugrulcetin17:07:52

@noisesmith

ERROR in (lower-upper-dual-bound-test) (Compiler.java:6808)
expected: (= "TEST_LEVEL := POSITIVE" (eval-from-fn-call (quote {MEASURED_VALUE? true, X 12}) (data->code data)))
  actual: clojure.lang.Compiler$CompilerException: Syntax error compiling at (/private/var/folders/gs/dtdlkj7551vbd_d7jl8dxfb80000gn/T/form-init14550509181402349056.clj:1:6420).

Caused by: java.lang.RuntimeException: Unable to resolve symbol: ccase in this context

ghadi17:07:06

well there's a huge clue

noisesmith17:07:13

OK - which file has ccase in it? grep could find it

ertugrulcetin17:07:14

some tests are failing not all of them

ertugrulcetin17:07:31

but works on REPL, which seemed to me weird 😕

noisesmith17:07:46

probably because you haven't reloaded the file after typoing ccase

ertugrulcetin17:07:13

This is the ns that test is using: x-mind-report-parser.core

ertugrulcetin17:07:37

(ns x-mind-report-parser.core-test
  (:require [clojure.test :refer :all]
            [ :as io]
            [clojure.edn :as edn]
            [x-mind-report-parser.core :refer :all]
            [x-mind-report-parser.util :as util]
            [x-mind-report-parser.queries :as q]))

ertugrulcetin17:07:41

all required all

noisesmith17:07:42

what I'm saying is ccase is not a clojure function, you either failed to define it properly or misspelled case

noisesmith17:07:49

that's your error

ertugrulcetin17:07:56

nope it's a custom macro

noisesmith17:07:03

OK then why didn't you require it?

noisesmith17:07:22

if you had required it, it would resolve

ertugrulcetin17:07:30

it's inside of x-mind-report-parser.core which is required in x-mind-report-parser.core-test

noisesmith17:07:07

are you trying to use it without the correct prefix?

ertugrulcetin17:07:41

Shouldn't it work after requiring in test file [x-mind-report-parser.core :refer :all] ?

ertugrulcetin17:07:31

I think leiningen has some weird thing, don't know

dpsutton17:07:16

if you restart your repl and run tests does it work?

dpsutton17:07:05

is your macro recursive by any chance?

💡 1
noisesmith17:07:37

oh yeah if it's recursive and didn't namespae the self call properly I could see this issue coming up

ertugrulcetin17:07:25

yeah it's actually

noisesmith17:07:26

you might be able to replicate by loading the test ns from user instead of your default starting ns

ertugrulcetin17:07:30

@dpsutton restarted new REPL session, loaded files to the REPL and ran run-tests all worked without problem

noisesmith17:07:12

@ertucetin if you use lein run -m clojure.main you can get a new repl with no readline etc. and nothing but clojure.core preloaded, from there try loading your test ns via require

noisesmith17:07:20

that might reproduce the error in a repl

dpsutton17:07:12

is this public?

dpsutton17:07:53

wow. didn't know that load-file was a special form

noisesmith17:07:50

wow, this was fun

Clojure 1.10.1
(ins)user=> (apropos "special")
(clojure.core/special-symbol?)
(ins)user=> (->> (all-ns) (mapcat (comp keys ns-publics)) (filter special-symbol?))
(& def)
(ins)user=> (source special-symbol?)
(defn special-symbol?
  "Returns true if s names a special form"
  {:added "1.0"
   :static true}
  [s]
    (contains? (. clojure.lang.Compiler specials) s))
nil
(ins)user=> (pprint (. clojure.lang.Compiler specials))
{& nil,
 monitor-exit
 #object[clojure.lang.Compiler$MonitorExitExpr$Parser 0x4bd1f8dd "[email protected]"],
 case*
 #object[clojure.lang.Compiler$CaseExpr$Parser 0x7ec3394b "[email protected]"],
 try
 #object[clojure.lang.Compiler$TryExpr$Parser 0x278bb07e "[email protected]"],
 reify*
 #object[clojure.lang.Compiler$NewInstanceExpr$ReifyParser 0x625abb97 "[email protected]"],
 finally nil,
 loop*
 #object[clojure.lang.Compiler$LetExpr$Parser 0xa4ca3f6 "[email protected]"],
 do
 #object[clojure.lang.Compiler$BodyExpr$Parser 0x3543df7d "[email protected]"],
 letfn*
 #object[clojure.lang.Compiler$LetFnExpr$Parser 0x3b2c0e88 "[email protected]"],
 if
 #object[clojure.lang.Compiler$IfExpr$Parser 0x549949be "[email protected]"],
 clojure.core/import*
 #object[clojure.lang.Compiler$ImportExpr$Parser 0x6da00fb9 "[email protected]"],
 new
 #object[clojure.lang.Compiler$NewExpr$Parser 0x3a3e4aff "[email protected]"],
 deftype*
 #object[clojure.lang.Compiler$NewInstanceExpr$DeftypeParser 0x3e10dc6 "[email protected]"],
 let*
 #object[clojure.lang.Compiler$LetExpr$Parser 0x5a7005d "[email protected]"],
 fn* nil,
 recur
 #object[clojure.lang.Compiler$RecurExpr$Parser 0x4189d70b "[email protected]"],
 set!
 #object[clojure.lang.Compiler$AssignExpr$Parser 0x1a7288a3 "[email protected]"],
 .
 #object[clojure.lang.Compiler$HostExpr$Parser 0x4044fb95 "[email protected]"],
 var
 #object[clojure.lang.Compiler$TheVarExpr$Parser 0x62e6b5c8 "[email protected]"],
 quote
 #object[clojure.lang.Compiler$ConstantExpr$Parser 0x1af146 "[email protected]"],
 catch nil,
 throw
 #object[clojure.lang.Compiler$ThrowExpr$Parser 0x4c398c80 "[email protected]"],
 monitor-enter
 #object[clojure.lang.Compiler$MonitorEnterExpr$Parser 0x60641ec8 "[email protected]"],
 def
 #object[clojure.lang.Compiler$DefExpr$Parser 0x75390459 "[email protected]"]}
nil
(ins)user=> (pprint (keys (. clojure.lang.Compiler specials)))
(&
 monitor-exit
 case*
 try
 reify*
 finally
 loop*
 do
 letfn*
 if
 clojure.core/import*
 new
 deftype*
 let*
 fn*
 recur
 set!
 .
 var
 quote
 catch
 throw
 monitor-enter
 def)
nil

dpsutton17:07:56

(doc &) is new to me

noisesmith18:07:20

wow - so load-file definitely isn't in that list of specials, but it's definitely not normal at all

(ins)user=> (source load-file)
Source not found
nil
(ins)user=> (source clojure.core/load-file)
Source not found
nil
(ins)user=> load-file
#object[clojure.lang.RT$3 0x64a8c844 "[email protected]"]

noisesmith18:07:54

also the back-story on clojure.core/import being a namespaced special form is fun

noisesmith18:07:27

when it was upgraded from a macro to a special form, they registered the form as fully qualified, so old code that used the full name to resolve it wouldn't break

ertugrulcetin17:07:40

@noisesmith (require 'x-mind-report-parser.core-test) returns nil if I move to run-tests to comment block, if move out, works and returns: Ran 8 tests containing 16 assertions. 0 failures, 0 errors.

noisesmith17:07:38

what about (run-tests) from the repl

noisesmith17:07:42

from your own ns

ertugrulcetin17:07:42

@noisesmith also works, here:

noisesmith17:07:39

I have no idea what that tool is or what namespace it is using to load your code, or what middleware it might be using

noisesmith17:07:05

I suggested running the command I did because for behavior this strange getting layers out of the way helps

dpsutton17:07:45

if you're running (run-tests) it means you're in the test namespace

dpsutton17:07:18

Defaults to current namespace if none given.

dpsutton17:07:25

i think you want to be in a namespace where ccase is not defined in that ns. your test file :refer :all so its defined there

noisesmith17:07:15

right, thus my initial suggestion of lein run -m clojure.main to create that set of conditions

👍 1
noisesmith17:07:37

then you'd use (clojure.test/run-all-tests) after requiring the test ns but not switching to it

ertugrulcetin17:07:07

lein run -m clojure.main -> (require '[clojure.test :refer :all]) -> (clojure.test/run-all-tests) -> {:test 0, :pass 0, :fail 0, :error 0, :type :summary}

noisesmith17:07:39

you didn't require your test ns

noisesmith17:07:55

also don't :refer anything

noisesmith17:07:11

the symbol not being mapped to this ns is part of what we are trying to recreate

ertugrulcetin17:07:50

lein run -m clojure.main -> (require 'x-mind-report-parser.core-test) -> (clojure.test/run-all-tests) -> {:test 8, :pass 11, :fail 0, :error 5, :type :summary}

ertugrulcetin17:07:01

got: clojure.lang.Compiler$CompilerException

noisesmith18:07:12

OK - so we've recreated the conditions of the failure, and it's likely what @dpsutton thought

noisesmith18:07:41

somewhere yo have 'ccase where you should have ccase` for example

ertugrulcetin18:07:52

hmm let me update

noisesmith18:07:04

being in a namespace that refers ccase (like the original ns, or your test ns) hides the error, as the symbol is referred during expansion if you are in the ns, but the error will occur if you have some other active ns when the code is expanded I bet

☝️ 1
ertugrulcetin18:07:38

wow that was hard one 😄

ertugrulcetin18:07:46

thank you so much @noisesmith @dpsutton, I'll try to update the code and make it work

dpsutton18:07:07

it should be clear if you macroexpand it

ertugrulcetin18:07:33

will do thanks

dpsutton18:07:55

(let*
 [x__1019__auto__ 1]
 (if x__1019__auto__ (clojure.core.cache/my-and 2) x__1019__auto__))
you see the my-and there. if it was an unqualified my-and that would be the bug. the expansion would only work in an environment where that symbol was defined

dpsutton18:07:26

(macroexpand '(my-and 1 2))

ertugrulcetin18:07:34

yeah makes sense

👍 1
bitpadawan18:07:00

is arcadia (unity for clojure) abandonware?

noisesmith18:07:32

last merge was april, that doesn't sound fully abandoned to me

noisesmith18:07:50

@bitpadawan also in my experience, mainstream clojure coding style (and the compatibility levels maintained by the vm and clojure itself across releases) means that libraries actually get "finished", or only change as features come in, not as an endless series of red-queen scrambling ot keep up with everything else's changes

☝️ 1
noisesmith18:07:12

arcadia is still alpha or beta, but it's being worked on

noisesmith18:07:44

also it helps that nobody writes the security software in clojure, you get those updates via the vm or downstream deps

bitpadawan18:07:44

i've noticed that, but as a beginner it's hard to tell the difference. it's also that i don't see much/anything on reddit, here, etc, and also few beginner-friendly resources etc

bitpadawan18:07:00

it's pretty cool that it exists

noisesmith18:07:04

that's fair - I don't know if arcadia is a very beginner friendly project (yet at least)

noisesmith18:07:39

@bitpadawan the tooling scenario for the clr port of clojure is in a meagre state

noisesmith18:07:10

it seems like you have people who are used to the clojure/cljs tooling who simply give up, and people who are used to the normal clr tooling, who have nothing new to document and they just work as usual

noisesmith18:07:27

that's what it looks like from here at least, I've only played with clojure-clr lightly

noisesmith18:07:34

it doesn't have its own package manager for example

noisesmith18:07:47

people just use the standard clr tools, or do deps by hand(?)

noisesmith18:07:24

you can of course use the executable to launch a repl (just like you can with jvm clojure), but most devs don't use that kind of workflow

noisesmith18:07:38

maybe that's part of what makes getting into unity odd?

noisesmith18:07:16

@bitpadawan the last time I saw the arcadia devs they were excited about a DSL for performant hot loop game code compiled from clojure

noisesmith18:07:22

called "magic" iirc

noisesmith18:07:33

but that's definitely not a newbie friendly project

bitpadawan18:07:31

you may be right, i haven't actually installed it yet, just poked around online. what confused me more was the unity-specific code examples, it's like it's assumed the reader already has done unity in c#

bitpadawan18:07:02

i sort of get the same vibe with cljs and react native, like you're supposed to know rn already and then things would make sense

noisesmith18:07:51

@bitpadawan yeah, jvm clojure is slightly better about this, but Rich does talk about intentionally making a tool for people who are already experts, rather than one whose use is obvious to a beginner

noisesmith18:07:21

which is definitely an unconventional approach these days

bitpadawan18:07:31

yes, the beginner-friendly documentation for clojnure has improved a lot over the last 1.5 yrs or so

bitpadawan18:07:46

when i got into it,most of it was very hard to follow

bitpadawan18:07:55

for a complete lisp beginner

noisesmith18:07:01

right, but it's not just a question of docs, there's a series of architectural choices you make differently if your intended audience are beginners

noisesmith18:07:45

like seamless platform interop - killer feature, not novice friendly at all

noisesmith18:07:04

most languages hide it behind hard to use syntax on purpose IMHO

bitpadawan18:07:21

true,, i've learned more than i ever wanted about java already in order to expand my clojure horizons

bitpadawan18:07:31

the same thing seems to be shaping up now about js / cljs in the journey

noisesmith18:07:10

yeah, it's really useful in terms of maximizing the time of the core team of devs, and making something that doesn't put a brick wall in your way once you need to do something a little odd

noisesmith18:07:17

but it's also not newcomer friendly at all

bitpadawan18:07:38

it's the search space being so huge and so many tools from other environments that makes it difficult to pick up i think, when you don't already have the background knowledge

bitpadawan18:07:28

frankly i've only stuck with it because i find clj/cljs so elegant

bitpadawan18:07:00

for a beginner it's harder than it should be, but it is what it is right now

bitpadawan18:07:49

the fact that i could in theory use clj/cljs to make mobile apps, desktop apps, backends, and SPAs/frontends is extremely appealing though

seancorfield18:07:34

I think Clojure/Script occupies an unusual space: complete beginners can pick up the core language very easily but struggle with the ecosystem as a whole (regardless of the host platform) -- and experienced developers can pick up the ecosystem fairly easily but can struggle with idioms (depending on how much "bad OOP" is entrenched in their background).

seancorfield18:07:03

Intermediate developers can struggle with both, I suspect.

💯 1
noisesmith18:07:28

it offers a great path to lead you forward to skill though (in my experience / opinion)

noisesmith18:07:48

it contains the right things to bootstrap yourself into better decisions

bitpadawan18:07:27

fwiw, you oldtimers with the skills, don't be shy about writing tutorials .. 😛

bitpadawan18:07:39

the more the merrier 🙂

bitpadawan18:07:43

that's a good observation @seancorfield

seancorfield18:07:04

Yes, and it can be a long, slow "climb". I was very familiar with the JVM when I got into Clojure a decade ago and even with past experience in FP and Lisp, it took me a long time to shake off decades of OOP practices and get comfortable with the idioms (I got started with C++ in '92 and Java in '97). And I'm still learning 🙂

seancorfield18:07:49

@bitpadawan The difficulty for me in writing tutorials is that I find it very hard to know what I'm assuming for an audience of beginners -- I've had several folks tell me I seem to skip over a lot of stuff that's important to them 😐

markbastian19:07:36

Question on the canonical way to package a cljc library… I have a library I’m working on that is pure cljc with the exception of needing a priority map. My project.clj depends on org.clojure/data.priority-map in the main dependencies block and tailrecursion/cljs-priority-map in the :cljs profile and the code uses reader conditionals to bring in the right code. Upon deployment, the concept of profiles are gone - it’s just a jar. Is there a “right” way to declare dependencies in this case? I could see a few options: 1. Making both :provided and explicitly state in the docs that you must require the one you need depending on your deploy target. 2. Depend on the jvm version and mark the cljs one as provided. 3. Make the both required dependencies. In this case your jvm uberjar target would bring in the cljs version as well. Thoughts?

phronmophobic19:07:42

The dependencies you've mentioned here both seem pretty small which means I would lean towards option 3. For this particular case, it seems like the cost of including both dependencies is small (relatively small extra dependencies) and that the benefit of including both dependencies is reasonable (less setup, less documentation explaining how to get started). If the dependencies were larger, I think the other 2 options would be worth considering. However, I'm also interested in how other clojurians think about this.

markbastian21:07:54

Thanks for the advice. Think I’m gonna go with #3.

bitpadawan20:07:27

> it's a bit of an art @seancorfield , thinking with a beginners mind, putting yourself in their shoes and from the vantage point of experience, cover the distance .. from a certain point onwards, for example assuming the reader has at least mastered the basics of the language

seancorfield20:07:23

@bitpadawan Yeah, not an art I've been able to master much, unfortunately. I'm better at writing documentation now than I used to be, but I'm not good at writing beginner-focused/introductory material...