This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-01-31
Channels
- # aws (1)
- # beginners (70)
- # boot (12)
- # calva (45)
- # cider (45)
- # clara (1)
- # cljdoc (10)
- # cljs-dev (133)
- # clojure (315)
- # clojure-dev (2)
- # clojure-europe (2)
- # clojure-italy (16)
- # clojure-nl (1)
- # clojure-spec (23)
- # clojure-uk (19)
- # clojurescript (48)
- # cursive (11)
- # data-science (5)
- # datomic (18)
- # figwheel-main (3)
- # fulcro (18)
- # graphql (14)
- # jackdaw (1)
- # juxt (1)
- # kaocha (1)
- # off-topic (10)
- # other-languages (3)
- # pathom (2)
- # pedestal (7)
- # re-frame (23)
- # reagent (1)
- # reitit (4)
- # ring-swagger (12)
- # rum (4)
- # shadow-cljs (26)
- # specter (6)
- # speculative (12)
- # tools-deps (44)
- # vim (8)
- # yada (2)
is there an existing library for converting a swagger.json file or json schema file into spec definitions? I know spec-tools supports spec to both formats, but I'm more interested in the opposite direction
Hello, does anyone know of a way to decode php laravel sessions in clojure?
FYI I couldn't find any documentation (outside of the clojuresque api docs :)) on how to use prepl so I wrote some https://blog.jakubholy.net/how-to-use-clojure-1.10-prepl/
Hi folks! My current task is to profile a Clojure app (with Aleph and core.async) and after googling for some time it seems that is not an easy task like in Golang... I need some metrics like "how many goroutines are running", memory, and cpu. Does anyone know a way to get these metrics?
@lucelios I don't know about any tools to get stats for core.async specifically, but for memory and CPU profiling there's the usual JVM tools like https://visualvm.github.io/
ehm, I used metadata on a fn like this:
(defn ^String foo ([] "") ([a] (str a)))
(set! *warn-on-reflection* true)
user=> (.length (foo))
0
Apparently that works too, although I can’t find it in the docs like this.you’re putting it on the var there, not the fn
because it’s using the var meta
it’s not reflecting
your use of “too” above seems like there is some prior conversation this is in reference to, but I haven’t read that
just mentioning that in case there is context I should be aware of
so “too” above implies there is something else that works, but not sure what that is
not your “too”, @borkdude’s “too”
yeah, it works, but I agree with bronsa that putting it on the ret type is better
not every arity has to have the same return type
(not that I would recommend having different return types)
it's just a matter of consistency, tagging return types on the argvec is guaranteed to work with any legal type hint and Do The Right Thing
there is a string of cases where tagging return types on the var doesn't do what you want, silently
I think it’s also the tightest scoping to your intent
Syntax error macroexpanding clojure.core/let at (REPL:1:1).
it’s tha core specz yo: https://dev.clojure.org/jira/browse/CLJ-2473
@slack.jcpsantiago https://clojuredocs.org/clojure.core/spit#example-542692c7c026201cdc326975
Anybody aware of a walk-through describing how reagent components that deref an atom register themselves as watching that atom?
@markw my memory is that r/atom deref itself does the registration, since the deref happens inside the component data literal
but I haven't touched reagent in ~year
@markw the magic is all here https://github.com/reagent-project/reagent/blob/master/src/reagent/ratom.cljs#L43
I think this is the line that literally answers your question: when you deref an ratom, your context is captured as a watcher https://github.com/reagent-project/reagent/blob/master/src/reagent/ratom.cljs#L145
or, more precisely: no walkthrough that I know of, but the code isn't that hard to read
@noisesmith thanks... yeah you're right I shouldn't be lazy and just read the source
well, hopefully at least knowing the precise place the behavior is defined is a good start
I could imagine it eg. being part of the mounting code or the component loading code
https://github.com/seancorfield/depstar
https://github.com/juxt/pack.alpha
https://github.com/luchiniatwork/cambada
I found 3 uberjar maker for deps.edn with clj
. Do we need 3 as community? I always like simplicity about Clojure tools and making choices. Which one is recommended?
I’m still using lein to deploy for some projects, next to clj, I guess you could use both
the clj tooling has been explicitly called "not a build tool" over and over again, and aot compiling seems like a clear cut build step (of course so does building jars)
the pack.alpha has some suggestions for adding aot compilation on top of it, but it seems pretty adhoc
cambada actually defaults to aot compiling everything because it larger looks it is for building graal-native stuff, which requires aot
no worries about that. there's a lein plugin to gather deps from deps.edn so you can just use lein for build tooling and deps.edn for what you use it for now.
it just seems like, if you have some weird corner case (building aot jars) and it is well served with lein, why make the change to clj?
i understand lein and deps.edn use subtly different dependency resolution so i'm not sure that your built artifact is guaranteed to reflect what you developed. Need to find out what that plugin does, if it resolves everything in the way tools.deps.alpha would do (and i suspect this is what it does)
And clj solve other corner cases easy like git dependency or local folder dependency
Why, specifically, do you not want a JAR containing .clj
source files? That's how we package everything for production deployment at work.
(we use my fork of depstar
to build uberjars from source, and run them with java -cp path/to/the.jar clojure.main -m entry.point ...
in production)
Why did you make depstar instead of use https://github.com/juxt/pack.alpha or https://github.com/luchiniatwork/cambada ?
Because depstar
does what I want and is nice and minimal. And @U050ECB92 made depstar
. I just forked it, fixed some bugs, and added the "thin JAR" creation feature.
Also, I'm pretty sure Cambada didn't exist when depstar
was created?
How does it work in the context of: https://clojurians.slack.com/archives/C03S1KBA2/p1548963422931200
Juxt's pack.alpha
came first (back in December 2017, it seems). Then depstar
in early 2018. Then Cambada. Based on initial commits, not public releases.
Not sure what you mean by that question... that's exactly what depstar
does.
Sorry if I am asking about something stupid. But does it only pack source files or compile them into different format? As I understand compile to different format. But hiredman wrote “just zips up the source code and calls it a jar” and it make me confuse.
A JAR file is just a ZIP file with the extension .jar
Not sure what you mean by "compile them into different format".
When you compile Clojure source .clj
files, you get .class
files. You don't need the .clj
file present if you have all the .class
files that it compiles into -- unless it does some action at runtime that would require the source (such as loading a specific source file). AOT is transitive (which is considered a bug by most people -- there is at least one open JIRA issue about it) which means it compiles the namespaces you specify and any namespaces that are reached by loading those namespaces (recursively) -- which means that if your AOT'd code uses require
/`resolve` at runtime on namespaces that were not part of the AOT pass, you still need their source files.
It depends on whether you configure them to AOT everything or just specific namespaces. And even at runtime, Clojure code can still pull in new dependencies (as source code) and load them (which compiles them into class images in memory).
I think it's a long-standing mistake that Leiningen chose to have as the default in its basic app template, a compiled main namespace and a :gen-class
directive.
Leiningen does a lot of stuff by default that is not necessary.
Licensing. As several people said.
It doesn’t protect source code in any way. It makes potential penalty if you can prove it.
ok so I assume you make source code public for all your apps (unless it is on cluster only, so they can’t read anyway)
All the code I have written for myself, that I own the rights to, has been open source -- for about thirty years. The code I write for companies, in general, is owned by those companies and it's up to them whether they release it publicly. Some code I wrote for Macromedia was open sourced (in the early 2000's). Some code I wrote for a client after I quit Adobe in 2007 was open sourced. Various companies I've worked for have open sourced some of their code. My current employer supports all of its developers contributing to existing open source projects -- and has allowed me to release a number of libraries as open source that were written internally, as well as having its own GitHub organization with some open source libraries.
We have some code at work that is proprietary in terms of containing internal business logic and/or algorithms -- but we wouldn't release that in compiled format. Nothing to do with source code.
> All the code I have written for myself, that I own the rights to, has been open source -- for about thirty years. I envy you have such opportunity to do it 🙂
A couple of our developers have given conference talks, going into some depth about how we use certain software and sharing code examples publicly.
The real reason most companies won't release their code as open source is that they don't want to have to maintain it as an open source project -- because that's a big commitment. We've had a lot of discussions about that internally.
For a long time, we only had one official company library available as open source, and it was on BitBucket (an internal repo made public), and that was all about the potential drain on our time for maintaining it for the community if we put it up on GitHub for "everyone".
With the primary delivery platform now being the web, there's no deliverable for backend code to clients, so compiling (to whatever format) is far less important. And of course all your client-side code is public as JavaScript and the entire world can read it.
Anything delivered as a desktop app built on Electron etc -- all your source code is public there too. Because JavaScript.
Companies like Microsoft are now releasing huge swathes of code as open source. Closed source is pretty much dead as a concept as far as modern companies are concerned.
> Closed source is pretty much dead as a concept as far as modern companies are concerned. That is interesting topic. I see this trend more and more, but I have never seen rationale for that.
PHP, Python, Ruby... all delivered as source code, regardless of whether the code is open source or restricted by licensing.
That's always been true tho' even with proprietary closed source systems -- look at how widely pirated desktop software from Microsoft and Adobe is...
Adobe has a specific software license enforcement team that work with law enforcement worldwide and they conduct raids on pirating organizations, as well as sue companies that use pirated versions of their products. I'm sure Microsoft, Oracle, et al all do as well.
But on the other hand they power is in number of users, also this who crack they system
Unless you're a giant mega-corp, with billions in revenue from closed source desktop systems, piracy is really not worth worrying about tho'...
I mean more people use they product, better for them, because more people will buy a licence.
Your ideas in software are not likely to be novel 🙂
There's much less true innovation in software than a lot of people seem to think.
So the software you produce -- there's almost no benefit to keeping it "secret" in the vast majority of cases.
Still if somebody will do copy & paste your system for which you spent a lot of money you will feel bad about that
If you really "spent a lot of money" and have a large amount of business riding on it, sue for license infringement. That's what licenses (and courts) are for.
And if it's closed source, they'll just crack the product anyway. It's no defense.
Really, if you think closed source vs open source is some magic defense against piracy/cracking you're delusional 🙂
Exactly, Russia and China et al will do whatever they want with your software regardless of whether you release the source or not!
Reversing situations: so why companies close they source code? Everything should be public.
Example: a Clojure open source library out there was made by a Russian as a copy of one of my libraries and they removed the license and copyright notices and passed it off as their own code. They announced the library on the Clojure mailing list and I called them out for license infringement -- and they recanted because they were caught in public breaking the rules.
Closed source is the old-fashioned model. Companies with old-fashioned thinking still follow that model. Companies still use COBOL, for example 🙂
This is an interesting article on the thinking behind keeping things closed: https://opensource.com/business/13/10/seven-reasons-closed-better-than-open-source
But for example turquoise model of managing business is new “fashion”, but I don’t believe it fits everywhere
You're welcome. As you can imagine, I'm very much an open source advocate, having been doing this for about thirty years 🙂
I can imagine. I would like to do more open source, but not so easy. Trends from US go to Poland after a few years 😉
Folks can always ... yeah, what @hiredman said...
bronsa built a pretty good decompiler from class files to clj in addition to the classfile -> java decompilers
sure, but decompile Clojure is probably much more difficult to debug, than Clojure code
There's no security provided by AOT compiling Clojure code.
(or compiling Java code)
I know, just sharing source of the code sounds risky for me and second reason: not all code is for free 😉
but, I mean, plenty of not free code is distributed everyday, right? that is what a license agreement is for
I don’t want to go deep into security topic, that was not the point of my question 😛
Software licensing is ... yeah, dammit, what @hiredman said! (again)
next thing, you’ll want the source code for voting machines
I'm sure somebody out there has a Classloader implementation that loads encrypted resources based on a key that needs to be present at runtime, I bet it's a pain to actually use though
I mean because code is public and everybody know what system cointain, we needs solutions like alpine
I am afraid if bank will publish they source code many people will lose they money 😉
don’t tell anyone, but banks use a lot of open source software
@noisesmith which is a key distribution problem not a source code problem
@kwladyka that is based on a deeply flawed understanding of the basis of computer security
they use legal means to fix the ensuing problems
if you look at crypto these days, no one wants to use a crypto system that relies on the implementation being secret, only on the keys being secret, they want everyone to know the implementation
So do you want to say me you publish source code of your all projects? Free and pay? I don’t believe.
legally I am unable to publish the source code to the stuff I do for pay because the rights are not mine
I am sure there is dependency between code publicity and number of trying hack. But it is besides of our topic.
there are reasons not to provide source code, but if you put security on that list you have failed to secure anything
I believe it reduce probability. Not making system secure. Just reduce attention and make things harder to hack.
But things are different for very popular public projects where security start to go on higher level.
I think it's pretty proven-out that open vs closed source doesn't have much bearing on security
plenty of businesses are based around JS applications that everyone downloads the source for ¯\(ツ)/¯
anyway, not saying that everyone should make their code publicly available. but that going to great lengths to obfuscate it is probably not worth the time/money
AOT is great for perf if you need it. as obfuscation from a business/security perspective, it's not that great and in the case of Clojure can be work to make sure all code deals with it well
There is no difference in performance since Clojure source code is compiled when it is loaded anyway and the JVM will do its JIT thing over time as the application runs -- AOT just saves that initial compile time on load.
yeah, considering that we’re doing AOT for each deploy, it kind of defeats the purpose (you only benefit from it once) - except that we maybe have a little less downtime between deploys
not always true, if you run app in kubernetes to do a job or create new containers when needed, then it is not once
I was referring to an app we run in production. In general (for libs, etc) you can benefit from it more.
For us, the biggest chunk of startup time is New Relic's instrumentation agent processing all of the classes as they load. The difference between AOT and non-AOT for us is noticeable but relatively small in comparison.
Just to summary up what I wanted to say and avoid misunderstanding: Not publish code doesn’t make code more secure. But publish code increase a risk to be hacked. For example bot looking known bugs in public code, which will notice hacker you use dependency with security issue, have overflow bug etc.
Anyone have a recommendation of a smalltalk written in clojure? I'm mainly interested in the "liveness" and "object protocol", not so much the parser. EDIT: removed incorrect statement regarding Clojure / Protocols
I removed in correct statement regarding Clojure / Protocols. Regarding original question -- is there any readable impl of Smalltalk in Clojure ?
re smalltalk, I think http://cloxp.github.io/ is smalltalk inspired
> But publish code increase a risk to be hacked. For example bot looking known bugs in public code, which will notice hacker you use dependency with security issue, have overflow bug etc. A good, relevant example here is Slack. The JavaScript code is not open source but it is public because of the way web apps work and it has a stupid exploit in it that allows hackers to bypass security checks in the client. Slack have known about it for years but refuse to fix it. If their source code was actually open source, you can bet they'd be deluged with pull requests to fix security issues like that.
That's why open source code is often more secure and less hackable than closed source code -- more eyeballs, more developers wanting to help fix vulnerabilities.
I agree about all popular projects. Just not sure if it will work the same if you will publish the system, which only you maintenance. For example small SaaS or system which you use only in company X. I think true is not black and white. But I agree. It is fair point.
I've spent the last hours hunting down an obscure compilation problem with a cryptic error message. I still don't understand it, but it seems that if I mark a (def q (into (sorted-set) [1 2 3]))
with a ^:const
, then other files that need q
for their (def)s
will not compile. And I now know that sorted-set
and ^:const
are needed. A vector won't cause it.
const means inline at point of use (that explains the scope of where you see the issue)
I don’t think sorted-sets can be const - they have an embedded comparator function that’s not representable as data
The error messages are… suboptimal.
Exception in thread "main" Syntax error compiling at (/tmp/form-init5251724450573718419.clj:1:73).
[...]
Caused by: java.lang.ClassCastException: Cannot cast clojure.lang.PersistentVector to clojure.lang.ISeq
https://dev.clojure.org/jira/browse/CLJ-944 is in the ballpark of this - it’s a really tricky corner of the compiler
This is way beyond me. I searched the bugtracker before asking, but I wouldn't have recognized this issue as related. Anyway, it isn't a major problem, it's just that finding the cause is very difficult.
can you repost the root cause of that exception chain (the last one one)?
Sure. I just abbreviated it, because it seemed rude to clog people's screens:
Caused by: java.lang.ClassCastException: Cannot cast clojure.lang.PersistentVector to clojure.lang.ISeq
at java.lang.Class.cast(Class.java:3369)
at clojure.lang.Reflector.boxArg(Reflector.java:552)
at clojure.lang.Reflector.boxArgs(Reflector.java:585)
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:132)
at clojure.lang.Reflector.invokeStaticMethod(Reflector.java:332)
at clojure.lang.Reflector.invokeStaticMethod(Reflector.java:325)
at clojure.lang.LispReader$EvalReader.invoke(LispReader.java:1322)
at clojure.lang.LispReader$DispatchReader.invoke(LispReader.java:853)
at clojure.lang.LispReader.read(LispReader.java:285)
at clojure.lang.LispReader.read(LispReader.java:216)
at clojure.lang.LispReader.read(LispReader.java:205)
at clojure.lang.RT.readString(RT.java:1874)
at clojure.lang.RT.readString(RT.java:1869)
at partsbox.preferences__init.__init1(Unknown Source)
at partsbox.preferences__init.<clinit>(Unknown Source)
... 34 more
The last 2 lines reference my file: it's the one that uses the ^:const
definition.to get this you have to think about what the compiler is doing. it’s trying to inline a value rather than look it up via the var.
it’s going to include that value in the class compiled for the function using the inlined value
I, and I am sure @alexmiller will disagree, would say never use ^:const
as far as I can tell, people reach for it because it has a similar name to things they use in other languages when they want things to behave correctly
@alexmiller Right. I understand why this happens in the including file.
but clojure already behaves correctly, and const almost certainly doesn't do what you intuitively expect
it has to store the constant value in a field of the class, but there is a limit to what you can store in bytecode. For simple values, this is easy, but iirc the fallback case is to pr the value to a string, that can be re-read back to a value in the consuming case.
sorted sets and maps don’t have a literal print syntax and no way to print/read the comparator function.
user=> (binding [*print-dup* true] (pr s))
#=(clojure.lang.PersistentTreeSet/create [1 2 3])
this gets into increasingly murky areas where reader eval is used
I’m not sure I’m getting 100% of the details correct on all this (it’s been a few years since I looked at it)
as @hiredman says, there are fewer places where this is worth doing than you might expect (I wouldn’t say none though)
in particular using ^:const in tandem with ^long or ^double can be useful for inlining primitive long or double values
if you don't say none, everyone wants to argue that they are special and should use, and they never should
In general, I really wish sorted sets and sorted maps with default comparators had a printable/readable representation. I run into this quite a bit, and I don't have a single custom comparator function.
As to this problem, what next? Should I work on a minimal reproducible test case, or is it unnecessary? It seems this is a known issue, and all that could possibly be improved is the error message, right?
I don’t think what you’re doing is a useful thing to do (that is, I can’t imagine how marking this ^:const well help you in any way)
so I would recommend just removing that :)
if you want to file a jira re the error message (or a clojure-site issue for better docs around ^:const), that seems reasonable
the docs around ^:const are, well, largely non-existent. there are a couple lines in the changelog when it was added
Seems like it’d be easy enough to copy and paste the lines from changes.md
over. Where would those go? I’ve seen this discussion come up enough that it’d be easier to just point someone to the site for reference.
There are also several other meta questions that come up too (myself and others have asked about ^:inline
).
the inline stuff is primarily intended for use inside Clojure itself and is not considered a public feature that we wish to encourage (it may be removed in the future)
https://clojure.org/reference/compilation is probably the best place to doc ^:const
if you’re adding your name, you need to complete the CA form (but if you’re fixing a typo or something, that’s fine)
Also, regarding ^:inline
, I’m aware that it is not intended for public consumption; my suggestion regarding it is to make that fact public in documentation so that folks (you included) don’t have to keep having the same discussions.
I did, I don't much care about the ^:const
. I just checked, and I have 253 ^:const
declarations. I use them mostly as a comment, really.
I'd suggest also maybe a re-evaluation of your relationship with sorted sets, if they are causing issues, maybe try and get a long without them (as a data point I am not sure if I recall ever using clojure's built in sorted sets outside of toy projects)
@alexmiller By the way, I really enjoy your weekly journal blog posts. If this doesn't take too much of your time, it is worth doing and is much appreciated.
I’ll +1 reevaluating sorted sets
I find using them means you are usually relying on order and linear searches rather than building indexes which will be a better solution
Hmm. They are actually quite useful in certain places. I use them for things like a list of quantities (build/order/whatever). A list like that has 1-20 numerical items. A sorted-set is perfect, as it lets me declaratively specify that it should always be sorted. I could stop using it, but then I have to worry about re-sorting in a number of places.
But — indeed if they cause more problems than they solve, I can just make sure I always sort manually.
What’s the complexity of operations in a sorted set, membership and insert and such? Is it worse than a regular set? (I got intrigued… :) )
they’re red-black trees, so mostly O(n log n) I think
Cool! I remember having quite a difficult time getting my head around RB trees when I got them in CS
The constant factors for Clojure's sorted sets/maps are typically a fair amount larger than sets/maps that are not sorted, e.g. n log_2 n is a constant factor 5 more than n log_32 n.
if you are sorting just for display (as a todo list or whatever) then you don't need sorted data, you need a sorted view over something
is it? seems like you run into the same issue as the sorted set. you can serialize its current order but how do you serialize the instructions for inserting new elements into the graph? that would be the comparator in the sorted set and would seem to be an analogous point for the graph insertion
is order a property of your data/information or is it a property of the particular data structure you are using to represent the data
(I would also agree about :const
earlier -- 90% of the time I've seen it in the wild, it is incorrect.)
1. sorted sets are cool, most of my favorite things are built on them to some degree or other (https://git.sr.ht/~hiredman/dset) 2. if something is causing you trouble stop using it 3. always have a third thing
I would humbly suggest that the word "incorrect" is too strong. ^:const
, at a first glance, seems like an indication that something is a constant and will never change. Marking things this way is entirely reasonable
As for sorted sets, of course I don't require them. I can make sure that every function that needs the data sorts it before using (and no, it's not just "a view", there are multiple views, there is also logic that assumes the data is sorted). It's a tradeoff between declarative and procedural.
but that's not how const works
@U0511PZ1R here’s a link to the aforementioned changelog entry explaining ^:const
https://github.com/clojure/clojure/blob/master/changes.md#215-const-defs
Well, as an aside, I do understand how const works. The inlining is not critical in the code that I currently write, but I am aware of it. What I wasn't aware of are pitfalls, e.g. that not every Clojure data structure can be treated this way.
no data clojure structure does anything useful with const - it's an optimization for primitives
some data structures also error, none of them benefit
@U0511PZ1R I was trying to be helpful. I didn’t see anyone send you a link, etc.
As for "useful", isn't it true that if initialization is expensive, it would happen at compile time and the result would be inlined?
This is the kind of the discussion that needs to be distilled into reference material so these discussions can just be collapsed into a URL.
I don't want to come off as too strident here - I think given what const means in other languages it is poorly named and under documented
if your form is at the top level of the file, it's initialized during compilation, in clojure compilation happens when you load the file
the file is loaded when making the uberjar, anything at the top level is done again on app startup runtime load of that class file, otherwise def with side effects in it would break when using AOT
so you aren't saving as much time as you might think for defs at least
eg. (def tracking (get-tracking-data (fire-missiles)))
- fires missiles when you make the uberjar, then does it again when you load the resulting class file in an app
it's surprising, but it's the least surprising choice - think of all the potential bugs when people use def on eg. a db connection pool, or an http server etc. (of course best practice for devs is never do anything stateful in def)
No, what surprises me is that the same is done when there is a ^:const
metadata flag. I would not expect re-evaluation then.
const doesn't really do anything with data structures, it enables a fast path for compiled code that uses the const var, if the const var is a primitive
Seems like it’d be easy enough to copy and paste the lines from changes.md
over. Where would those go? I’ve seen this discussion come up enough that it’d be easier to just point someone to the site for reference.
There are also several other meta questions that come up too (myself and others have asked about ^:inline
).