This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-06-29
Channels
- # adventofcode (8)
- # babashka (12)
- # beginners (91)
- # calva (6)
- # cider (2)
- # circleci (11)
- # clj-kondo (19)
- # clojure (202)
- # clojure-australia (7)
- # clojure-brasil (1)
- # clojure-europe (123)
- # clojure-nl (2)
- # clojure-poland (24)
- # clojure-uk (6)
- # clojured (1)
- # clojurescript (91)
- # core-async (23)
- # cursive (16)
- # data-science (5)
- # datomic (26)
- # emacs (27)
- # events (2)
- # graalvm-mobile (50)
- # graphql (4)
- # honeysql (4)
- # instaparse (33)
- # lsp (24)
- # meander (22)
- # nrepl (3)
- # off-topic (26)
- # pedestal (1)
- # re-frame (15)
- # releases (1)
- # sci (1)
- # shadow-cljs (20)
- # tools-deps (22)
I would start by writing a for expression that destructs the input, and generates paths and values, then just reduce with assoc-in
(reduce
(fn [m v]
(assoc-in m (pop v) (peek v)))
{}
(for [[a-or-b m] {:AAA {:prop1 ["A"] :prop2 ["a"]}
:BBB {:prop1 ["B"] :prop2 ["b"]}}
[p1-or-p2 m] m]
[p1-or-p2 a-or-b m]))
@grzegorz.rynkowski In your output, are you sure you want the nested structure to be a vector, or should it be a map?
for vectors the same, just more annoying to construct, but always going through for to tear things apart
hi how can i know if something is set up for lazy evaluation (versus total realization) ? i assume some functions are lazy while others are not... is there a good resource I can use to find out which is which?
phrased differently, if i have a large dataset, how can i ensure i'm consuming one input at a time from the dataset, as opposed to lining it up for total eval
It's hard to give a generic answer since the input data format, the output data format, the type of data processing, and the hardware specs of the target computer(s) can all influence the design.
The only thing lazy is lazy is lazy seqs, anything not lazy seqs in -> lazy seq out is not lazy
And lazy seqs may not be strictly lazy, they may sometimes chunk up operations to improve efficiency
It's hard to give a generic answer since the input data format, the output data format, the type of data processing, and the hardware specs of the target computer(s) can all influence the design.
I run clj -X hello/run reprot error :Execution error (FileNotFoundException) at clojure.run.exec/requiring-resolve' (exec.clj:31). Could not locate hello__init.class, hello.clj or hello.cljc on classpath. I'm going to follow the code here https://clojure.org/guides/deps_and_cli
did you create src/hello.clj
as stated on this page?
looks like you running clj -X hello/run
from src
directory but you should run it from hello-world
clj
command expects to have deps.edn in the same directory where it is invoked
I have another question. I Include a local jar on disk. and I use (:require [libs/pdftools :as pdf]), when I press C-c C-k compile file report error:
require expects a namespace not a file
I don’t know what is inside of pdftools to suggest what to put into require.
sure, but the content of that jar should be either compiled java classes or clj source files. depending of type of the content you should use either :require to “require” clojure namespace or :import to load compiled java classes
in your last screenshot you have wrong argument for second :require statement (btw. this is an antipattern in organizing clojure namespaces, normally you should have single :require in one ns declaration)
could you show what is inside of that jar?
jar -tf path/to/file.jar
ok, so there are some java .class files then you should add into your ns something like this
(ns
...
(:import (org.pansome.pdf.common PdfUtils)
(org.pansome.pdf.common FontUtils)
...))
that means you are “importing” java class to make it accessable as PdfUtils
, FontUtils
etc.https://clojure.org/reference/java_interop this guide might be handy
(org.pansome.pdf.common PdfUtils)
note the difference - in my form there is no dot between common
and PdfUtils
emacs? looks like screenshots are taken in intellijidea
@U04V4KLKC Emacs 27 with tabs and LSP activated makes it look a lot more like Intellij...
@U025AG2H55F there is an #emacs channel for help. You might want to say what command you are running to help understand what you are doing in Emacs. The key binding could be doing anything...
Generally you need to evaluate things in order, so ensure the namespace has been evaluated in Emacs with Cider, which makes the functions from the required libraries available to the rest of the code.
thanks @U04V4KLKC
Clojure is amazing but Its hard to learner than other programming language!:hugging_face:
Is it hard because you are more familiar with other programming languages? Many language are based on the concepts from C, meaning they have obvious similar concepts. Clojure is a variation of Lisp, so prior knowledge of lisp makes Clojure trivial to learn. The issues raised in this thread are related to the environment rather than the syntax of the language itself (which is very small).
I'm trying to use this thing: https://clojars.org/cljsjs/rangy-textrange
Steps I've taken:
- added cljsjs/rangy-textrange {:mvn/version "1.3.0-1"}
to deps.edn
under :deps
- added [cljsjs.rangy-textrange :as rangy]
to my ui.cljs
under :require
- restarted my compiler and confirmed that rangy gets installed:
shadow-cljs - starting via "clojure"
Downloading: cljsjs/rangy-textrange/1.3.0-1/rangy-textrange-1.3.0-1.pom from
Downloading: org/clojure/clojure/maven-metadata.xml from
Downloading: org/clojure/clojure/maven-metadata.xml from
Downloading: cljsjs/rangy-core/1.3.0-1/rangy-core-1.3.0-1.pom from
Downloading: cljsjs/rangy-core/1.3.0-1/rangy-core-1.3.0-1.jar from
Downloading: cljsjs/rangy-textrange/1.3.0-1/rangy-textrange-1.3.0-1.jar from
but now when I go to compile it, I get the following error:
The required namespace "cljsjs.rangy-textrange" is not available, it was required by "sheluchin/ui.cljs".
Can anyone explain what I'm doing wrong?According to https://github.com/cljsjs/packages/wiki/Using-Packages#a-quick-javascript-interoperability-refresher, you shouldn't use :as rangy
. Simply require cljsjs.rangy-textrang
, then use js/some-fn-from-rangy
to call into it.
I'm not 100% sure your problem is related to this, because I don't get the same error. However, when I do as the link suggest, everything seems to work fine. EDIT: Nevermind, I didn't catch you were using shadow-cljs 🙂
This may also be relevant: https://shadow-cljs.github.io/docs/UsersGuide.html#_using_npm_packages
I’m trying to process a 2mb xml with xml-seq
parsed with clojure.data.xml/parse
with 1Gb heap — I’m getting
Syntax error (OutOfMemoryError) ,,,
Java heap space
with higher heap it works and it is fairly fast. it boils down to the below — any obvious thing I’m doing wrong?
(->> (xml-seq response)
(filter #(some? (seq (:content %))))
(keep (fn [e]
(when-let [id (get-in e [:attrs :id])]
[id e])))
(into {})))
you could refactor ->> to transducer equivalent to avoid creation of intermediate collections
(into {}
(comp (filter #(some? (seq (:content %))))
(keep (fn [e]
(when-let [id (get-in e [:attrs :id])]
[id e]))))
(xml-seq response))
the semantic is different, not sure if it is possible to replace with single operation
thanks, trying this out — but what it the actual issue here? is this somewhat expected?
(into {} (comp (keep (fn [e] (if (some? (seq (:content e))) (when-let [id (get-in e [:attrs :id])] [id e]))))) (xml-seq response))
every seq op is essentially a new chain of seq cells over the input (this is mitigated somewhat by chunking)
transducers remove all intermediate seq cells and produce only the output collection, which is particularly good if you are going to hold the entire output in memory
this definitely seems like a case where transducers would reduce memory footprint
the in-memory object representation of that 2 mb is probably a lot bigger (and then there are multiple seq wrappers around each node)
other aspects: 1) some? is not needed 2) (get-in e [:content :attrs :id]) instead of (get-in e [:attrs :id]), right?
well seems a good opportunity to learn transducers..
in the meanwhile, with clj-memory-meter
, the retained size of the whole xml is:
(mm/measure (xml/parse-str (slurp "/Users/bani/tmp/loadgroup.xml")))
=> "3.8 MiB"
is that fully realized or lazy?
I guess that's realized
(mm/measure (doall (xml-seq (xml/parse-str (slurp "/Users/bani/tmp/loadgroup.xml")))))
=> "26.6 MiB"
well, might be something else going on then
tools like yourkit can take heap snapshots and diff heap snapshots which I find to be pretty instructive in understanding leaks
they also have a "path to root" which can tell you why something is being held
[turned out i actually fixed this issue and encountered a similar one with a bigger xml (30mb) — apologies — that didn’t even parsed with 512mb. in that case, i only needed to count some elements, and eventually went with plain java stax route. thanks for the help!]
favorite lib for assertions / testing ?
for testing data structures https://github.com/nubank/matcher-combinators is really nice
I’m finishing my read of the Brave Clojure and would like to know where to go from here. I started a small project putting the things I learned to work and would like to know more about the following topics: • Server-side clojure, or how to make a rest api with it • Deploying clojure, specially on Heroku. Any help or recommended readings would be nice, thanks!
I found this documentation helpful to deploy a very simple app to heroku. https://devcenter.heroku.com/articles/getting-started-with-clojure?singlepage=true
https://practical.li/clojure-webapps/ may have some interesting pieces for you. Also take a look at metosin/reitit as a data oriented alternative to compojure for routing
The Luminus framework is a good start as well. Great docs. It's by the same author as Web Dev book mentioned by @U01EFUL1A8M https://luminusweb.com/docs/guestbook
Here is an example of a backend REST API whose contains all Heroku deployment config, etc. Maybe it can help: https://github.com/prestancedesign/todo-backend-reitit