Fork me on GitHub
#beginners
<
2020-07-10
>
Frosku16:07:58

Does anyone have any resources on how to create federated or p2p applications?

mathpunk16:07:45

if i have a nodejs file can i just read it into my repl as code? or does it have to be packaged somehow to do that?

noisesmith16:07:52

you can require / load it via the mechanisms node offers - node has its own version of require with its own rules about where files need to be

noisesmith16:07:06

and for a js file you can use that API directly instead of the cljs layer

noisesmith16:07:52

@mathpunk if you will be running with node as your js platform, this should be easy to translate to cljs https://nodejs.org/en/knowledge/getting-started/what-is-require/

noisesmith16:07:59

it's not compatible with the browser of course

mathpunk16:07:47

cooool thank you

dabrazhe18:07:37

[1 2 (range 3 5)] vector resolves as expected to [1 2 (3 4)] '(1 2 (range 3 5)) , i.e. escaped list, resolves to (1 2 (range 3 5)) which is suprising, because range is not eval'ed. I tried to escape it with or @ but it does not make much sense to me, nor does it help either.

Frosku18:07:04

Quoting keeps things from being eval'd, including their internal contents

Frosku18:07:14

What are you trying to get as an end result?

noisesmith18:07:12

if you use ` instead of ', you can use escaping

dabrazhe18:07:20

Practially I can use the vector, but I just want to undesrtand if there's a way to eval the (range) in this context

noisesmith18:07:25

you can also just use list

noisesmith18:07:49

(list 1 2 (range 3 5)) - the common newcomer mistake is to think ' is a thing for making lists

noisesmith18:07:33

you can also use eval explicitly, but the real fix is not preventing the initial eval in the first place, and ' is the thing that prevents eval

dabrazhe18:07:35

yes!, the ` quote works as expected : )

dabrazhe18:07:30

i.e. with escaping: (1 2 ~(range 3 5))`

Daniel Stephens18:07:46

I'm having to interop with some java code, just found that HashMap is not considered a map so I need to turn it into one. Seems I can do (into {} (HashMap.)) and that seems to work, but wondering if there is a preferred way (ideally) that would avoid duplicating the whole map?

noisesmith19:07:33

clojure doesn't copy the HashMap key and vals, it only creates new map-entry objects to put each pair in

noisesmith19:07:02

there's very little implicit copying in clojure, as idiomatic code assumes that values are immutable, so copying defensively is rarely needed

noisesmith19:07:36

also, you can use get on a HashMap, it's just things like conj assoc and dissoc that require clojure maps

noisesmith19:07:41

(ins)user=> (def hm (java.util.HashMap.))
#'user/hm
(ins)user=> (.put hm "a" 0)
nil
(ins)user=> hm
{"a" 0}
(ins)user=> (merge {"b" 1} hm)
{"b" 1, "a" 0}
hm is a java HashMap, the final result is a clojure map, no explicit conversion was needed

noisesmith19:07:22

(merge used get under the hood, that was fine, didn't require any copying, it does create new map-entry when merging, there's no way not to do that, java class or no)

Daniel Stephens19:07:47

Thanks, I wanted to use keywordize-keys but it failed to change anything since it asks if it's a map, though I've just realised I'll have to also recurively into it first to make it work since it appears to be a map of maps

noisesmith19:07:20

yes, that makes sense - but also, keywords on input data are often a bad idea

noisesmith19:07:50

use keywords if you are dealing with data literals from code, of course, but in many cases its simpler and less error prone to keep the original data type of the keys

noisesmith19:07:59

@ULNRSUK8C and to be clear, that into call doesn't copy the map contents, just the skeleton (though in this case you do need to do so recursively)

noisesmith19:07:25

if you fear a performance bottleneck, you can keywordize within the into itself, instead of walking/reconstructing the key twice

Daniel Stephens19:07:34

it's just unfortunate that it doesn't fit the same pattern as everything else here if I don't keywordize. I'm getting the value from a kubernetes client api and most come back as beans so I have used clojure/java.date from-java which gets me keywordized maps. In this case it's a custom resource so I get the map of strings to maps/strings

Daniel Stephens19:07:12

To be honest I'm not sure performance is any issue, I was just wondering if there was a more idomatic way of doing it, so I think I'll do as you say with the walk->into

noisesmith19:07:19

(defn clojurify [m]
  (into {} (map (fn [[k v]]
                  [(if-not (string? k)
                       k (keyword k))
                   (if-not (instance? java.util.Map v)
                      v
                      (clojurify v))])
        m))
something like this would help avoid the double reconstruction

noisesmith19:07:35

(first draft, probably has some shallow bugs)

Daniel Stephens19:07:22

thank you, that's very helpful! 🙂

Daniel Stephens19:07:32

spearate question, why do you prefer if-not in the above? Is it just a style choice to have the recursion last or something?

noisesmith19:07:39

it would probably be better code if you had a seq of predicate/key-transform pairs and a seq of predicate/val-transform pairs, then you have something reusable for foo->clojure conversion

noisesmith19:07:00

my first preference is for the first branch of a conditional to be the short one

noisesmith19:07:14

also, to put recursion last, which tends to be a side effect of that

noisesmith19:07:22

which makes recurring on the wrong branch stand out

noisesmith19:07:34

or failing to simplify while recurring, also stands out if you follow that rule

noisesmith19:07:02

especially since we only allow recur on tail calls, it's intuitive that a tail call should go as far down and to the right as possible

noisesmith19:07:13

it's kind of a lispy rule of thumb thing

Daniel Stephens19:07:49

makes sense, hadn't thought about the branch length, the recursion symmetry is nice

noisesmith19:07:05

another aspect: humans construct a stack as they read forms

noisesmith19:07:21

if the short form comes before the long form, you do less work moving up and down the stack :D

noisesmith19:07:45

(Germans likely have more practice with stack operations and are less limited by this issue)

Daniel Stephens19:07:39

😂 cheers for the help

g19:07:31

what’s the best way to ensure atomic channel writes? is wrapping it in an atom a good idea?

g19:07:58

or should i connect it to a persistent queue and keep checking if something is available

noisesmith19:07:49

what kind of channel?

noisesmith19:07:17

putting side effecting operations inside an atom is the opposite of a good idea

g19:07:42

k, figured

noisesmith19:07:44

"atoms" ensure data consistency but do not lock, instead they run operations speculatively and retry if there was concurrent modification

g19:07:45

like an async channel

noisesmith19:07:55

as in clojure core.async?

g19:07:59

yessir

noisesmith19:07:20

if so, channel writes are already atomic (unless you mean something different than I do by atomic...)

g19:07:53

yes, sorry - my writes are byte-level, and i’m thinking about ‘message’-level atomicity

g19:07:15

maybe that’s just a bad idea to begin with

noisesmith19:07:55

if the bytes need to go together, they probably shouldn't be mixed on the same channel as other bytes

noisesmith19:07:10

or they should be tagged so they can be separated and separately consolidated

noisesmith19:07:21

you can't block other users of a channel

noisesmith19:07:52

(I mean, you can do it accidentally by clogging up all go blocks, but even then you haven't blocked usages of channels outside go blocks)

g19:07:18

ok, yeah, i definitely need to rethink that. thanks for talking through it with me

noisesmith19:07:53

one idiom:

(async/thread
 (loop [acc []]
   (let [x (get-from-place)]
     (if-not x
       acc
       (recur (conj acc x))))))

noisesmith19:07:18

runs in its own thread, because IO inside go blocks just clogs things up and doesn't benefit from core.async

g19:07:06

oh, cool, was not familiar with thread

noisesmith19:07:51

but maybe you want to group every N bytes and send to a channel (use put! for that, pass in a channel when creating the thread)

noisesmith19:07:04

there's a few variations, but that's a good template for getting IO data into core.async

noisesmith19:07:21

@gtzogana also, since this is #beginners - consider that you can make everything a lot simpler and running (future (some-code that returns all contents)) and then calling deref later to get what it found

noisesmith19:07:50

core.async makes coordination simpler, but often the even simpler thing is to not need coordination

Daniel Matz23:07:20

Hi, everyone. I'm working on migrating a hobby app I have on Google App Engine to Clojure/ClojureScript. I was able to deploy a simple web app by creating an uberjar with https://github.com/tonsky/uberdeps. But when I try this with a more complicated app, I get a strange error that I don't understand:

Syntax error compiling at (clojure/tools/reader/reader_types.clj:1:1).
No namespace: clojure.tools.reader.impl.utils
I can use the same uberjar to launch the app locally, this error only happens on App Engine. Also, I confirmed that clojure/tools/reader/impl/utils.clj is in the uberjar. Any ideas? Thanks!

Daniel Matz00:07:55

I've been struggling with this for a little while, but of course the moment I post it, I figure it out. I think I had insufficient memory on my App Engine instance. I tried adjusting the -Xmx and -Xss flags, but ended up just bumping up to an instance class with more memory, and things seem to be working.

adam00:07:28

Just curious, how much was the RAM before and after?

Daniel Matz00:07:17

I was originally using the free F1 instance, which has a 256 MB limit. I had set -Xmx128m -Xss256k, but I was getting stack overflows, so I bumped it up to -Xss512k. When I switched to an F2 instance, which has a 512 MB limit, it works, and seems to be using about 300 MB of RAM. Any tips for keeping memory usage down?

jumar03:07:38

Don’t use many/any dependencies

jumar03:07:31

Limiting stacks isn’t going to help much unless you have many threads

jumar03:07:14

And 128m isn’t reasonable for a Clojure app in most cases - 256m is minimum; likely more if it’s something at least slightly serious (your app might work but you will pay a performance penalty because of limited memory)

Adrian Smith21:07:16

I imagine doing a native compile using graalvm would reduce the memory usage

jumar09:07:54

That's better reserved for simple (command line) tools rather than a standard approach of doing things

practicalli12:07:24

Heroku free tier uses 512 container images. This is how I deploy a deps.edn project on Heroku free tier (via CircleCI) https://practicalli.github.io/clojure-webapps/projects/status-monitor-deps/deployment-via-ci.html