postgresql is portable, my clojure project is portable, using postgresql and R. all developing Work in U disk.

in cljs, when I save a file in src/, a recompile is triggered: in clj, assuming I'm using component / writing reloadable-safe code, is there a way to say: on file save, reload all related modules?

@qqq IIUYC it's usually implemented using clojure.tools.namespace, e.g. take a look at https://github.com/weavejester/reloaded.repl

@metametameta: looks like I did not explain my problem well.

You have used clojurescript right? It's genrally something like "boot watch cljs" - and then, any time you save a *.cljs file, a recompile is triggered.

I want components to auto-reload whenever I save a *.clj file. Reloaded.repl appears to help writing components, but not do "reload on *.clj file being saved."

So you want to reload code in the already running app on src changes, right? In CLJS I use Figwheel for it. But when using Clojure you usually have to start REPL first and then invoke a helper function which starts the app, watches for changes in namespaces and reloads them:

lein repl
user=> (reset)

this pattern was originally explained in this article: http://thinkrelevance.com/blog/2013/06/04/clojure-workflow-reloaded

reloaded.repl lib just allows to not re-implement this same pattern in every project

@qqq if you use cursive you can use "sync files in repl", but nothing automatic "on save" exists unless you write watcher code.

There is ns-tracker.core that can watch file for changes.

@qqq Here's an example project that does auto-reload of both server-side clojure and client-side CLJS, including auto-restart of the app if specific server files are changed: https://github.com/danielsz/holygrail

Can somebody suggest a way to get the option --no-foo (foo being the flag) to work with clojure/tools.cli library? How should I declare the foo CLI-parsing spec?

@thosmos: noted; thanks!

is there a performance penalty to try-catch blocks .... EVEN WHEN NO EXCEPTION IS THROWN ?

Can plumatic/schema define specs for macros?

@qqq https://shipilev.net/blog/2014/exceptional-performance/

@rauh: I'm confused, what I read on SO says "try is cheap, throw is expensive"; but the blog post you're linking to suggests otherwise

@qqq Using exceptions is neglibile in performance, even faster than using a normal return value for error condition. As long as your exceptions only happen "exceptionally"

So no more than once every 1k...10k calls.

so here is my XY problem:

Using them for control flow and throwing them all the time is abig nono

I'm implementing a database, with optimistic concurrency

there's alost of [:swap old-val new-val]

now, when old-val is out of date, I'm considering throwing an exception to have the entire transaction fail

is this "exceptional case" or is this 'control flow' ?

what’s a succinct way to write a function that always returns its first argument?

imagine that a transction = a list of [:swap old-val new-vale] triplets

wei: first

@wei: first (builtin), or, alternatively (fn [x & rst] x)

Well how often do they occur?

actully wait, dumb question on my part

in generall, isn't the cost of setting up a transaction much higher than that of throwin gan exception ?

i.e. the fact that the transaction got rejected and I need to resubmit it -- that should drastically be more expensive than the throw/catch ?

@qqq first isn’t quite right, but (fn [x & rst] x) works

@wei: yeah, first returns first leemnt of first arg, not first arg; good call

@wei First is lazier than the desctructing FYI

@rauh: I think @wei's point is that first = wrong

Oh duh, yes, sorry didn't read properly

`(defn two-params [x y] (str "Two parameters! That's nothing! Pah! I will smoosh them " "together to spite you! " x y)) (two-params "b" "r"`

and that gives me this: CompilerException java.lang.RuntimeException: Unable to resolve symbol: two-params in this context, compiling:(C:\Users\zivan\AppData\Local\Temp\form-init3271026095323065387.clj:1:

why is it `(defn ... instead of (defn ?

I tried to add this snippet as a code in slack

with `

you want three `'s

haha thanks

😄

PSA: There is a slack setting that you can press enter without submitting when entering three `.

Anyone knows why closure can't find that function in that context?

Did you evaluate the fn definition?

Are you doing this in a text editor with a repl connection?

As rauh said, you'll need to evaluate the defn, or reload the namespace

I have two windows, text editor and repl connection in emacs

I just called main function

for returns a lazy seq right? how do I take the or or the and of a lazy list?

oh, I want this to short circuit, so to read as little of the lazy setq as possible (up to chunking)

qqq: You'll want to use some

or -> (some true? ...) and -> hmm, a bit more difficult

and -> (every? identity ...)

Hi #clojure!

With clojure.test, (use-fixtures :once ...) wraps a test namespace in a set-up and teardown fixture. I wonder if there's a way to do that for all tests that run during lein test

My issue here is that I have a fixture that is long to start and it's no problem for me to have it live accross several tests

So, https://stuartsierra.com/2016/05/19/fixtures-as-caches discusses my exact problem

I'd say the only issue I see here is that the fact that it relies on a separate -main function would break lein test, but I suppose I could live with that

@pyr perhaps there is a way to run the tests via API and then create your own alias which starts the environment prior to running the tests

@acron I'm angling towards this

with an entry in the :aliases map in lein

@pyr I used global flag and :once fixtures to achieve the goal of initing the db for the whole test suite. So that each test ns has (use-fixtures :once with-database) but it would skip db construction based on global flag value, simple.

the flag must be set on the first fixture execution

I don't know about this global flag

ah, it's a global var that you define yourself

ah, understood

for me it's the teardown flag that I need to only run once

but indeed I might be able to play with something like this

what makes the teardown in your tests so costly?

I start an embedded cassandra server

in the same JVM than my project, for test purposes

both set-up and teardown are time-consuming, and there's no need to do that multiple times

I see

the alias approach should be good enough

but it will probably run the server in the separate jvm then

well, dunno, guess it depends on how the alias will be implemented 🙂

@metametadata I meant an alias that wraps a call to run-all-tests, as mentioned in the article above.

ah, got it

I am wondering. What is the best way to “read ahead” from a lazy sequence where the realization (reading from some API for instance) takes the time? I thought seque was the way to go, but it seems that it holds onto head and I am getting OOM errors, that don’t happen without seque.

I guess you could use partition-all and create chunks that you realize one at a time and consume one by one

note that lazy seqs are chunked so it might realize the seq 32 elements at a time

depends how they are created

not all lazy seqs are chunked

Hey, I have a bit of an advanced question. I was wondering if it is possible to override the local bindings that result from a closure?

e.g.

(defn triple-adder-fn [a b] (fn [x] (+ x a b)))

(def triple-adder (triple-adder-fn 1 2))

(triple-adder 3)
;; => 6

(binding ['a 5
          'b 6]
  (triple-adder 3))
;; => 14

Where the last expression doesn’t actually work but I want to know if there was some way to do it?

The reason I ask is because in my use-case I don’t actually have control over the triple-adder-fn so I can’t just re-call it.

No that's not possible.

with volatile! - are reads from more than one thread safe? - I recall that the point of java's volatile annotation is that it guarantees using the value from memory and not caching the value in a register that could go out of sync with memory, so that would seem to indicate yes...

what the thread is doing is what determines if it is thread safe

let's assume the reads are not used as a basis of further writes to the same volatile

Volatile really should only be used from one thread or when there is a memory barrier between thread switches.

Otherwise I'd use an atom

OK - so not even for reads - I heard something about the reads from other threads being needed for transducers but didn't really understand

Right but you don't know when writes will be flushed, so a write to a volatile may be cached until the next barrier

so this would come up with transducers on channels, I assume

So this works in core.async because channels transfer controls via locks and the like

OK - it's a bit clearer now, thank you very much

Hi!

I want to ensure that a parameter passed to my function is actually from a certain namespace. Say I want to call (require [my-error-codes :as e]) (validate-error e/MY-ERROR-123) and then do something like (require [my-error-codes :as ex]) (defmacro validate-error [error] (assert (= (namespace error) 'ex)). This code does not work, but I would like to know how to express that properly.

urzds vars have namespaces, most values in a namespace will not

What I am actually trying to do is to write some kind of "registry" for error codes and later make sure that all error codes my application passes to its users is present in that registry. The variant with def-ing them all in a central namespace and then checking the namespaces of the error codes leaving the application was the first solution that came to my mind.

#'e/MY-ERROR-123 will have a namespace for example

e/MY-ERROR-123 (without the var-quote) only has a namespace if you use a type that is namespaced

perhaps what you want is to use (or create) a namespaced type here?

@noisesmith What exactly do you mean?

wait, not even vars are namespaced

urzds to test the namespace of a thing, it needs to be a thing that owns such a property

oh - is this why it's a macro, to catch the symbol used?

Yes, that's what I was trying.

oh, right

But indeed I could also define a type and then create instances of it on the fly... if that's somehow possible. Actually I've written Clojure code for some months now and never had to create any types or classes or things like that...

isn't what you really want here data validation?

eg. what spec or plumatic/schema offers

hehe, yeah, weeeeellll...

What I am actually trying to do is validate against an s/enum. But I need to find all the enum values, which are scattered throughout the code.

haha, something like (->> (all-ns) (map ns-publics) (mapcat keys) (filter valid-error?) (set))

That's why I wanted to mark all possible enum values in some way (in this case by making them symbols from some special namespace), so I can then gather them, construct a set, and use that to construct the schema.

That would work, too. Sounds like an interesting approach.

it's basically how clojure.test finds tests

Maybe I should look at clojure.test for inspiration then... 🙂

Thanks!

that still seems hacky to me though - but maybe it makes sense for your code

I don't know. How do people usually do this? Ensure that error codes leaving e.g. their HTTP API can be looked up somewhere?

Or do people just throw random strings at their users and don't care about them following any pattern?

(reading manual on MapDB) Hereafter is a simple example. It opens in-memory HashMap, it uses off-heap store and is not limited by Garbage Collection. ^^ I don't get it. How is something IN-MEMORY yet OFF-HEAP and not affected by GC ?

them=the strings

urzds libs like liberator and ring-swagger take a different approach, by explicitly describing the error cases (or using the ones that REST documents implicitly) as I understand it

@noisesmith Do you happen to have a link at hand, that describes how they do it?

@urzds HTTP has a pletora of status codes to communicate the semantics of an error (https://en.wikipedia.org/wiki/List_of_HTTP_status_codes)

@qqq by allocating OS memory outside the vm, I think

@urzds I would start with the github pages for liberator or ring-swagger - both are meant to organize a clojure web api

@urzds and there is https://tools.ietf.org/html/rfc7807

@ordnungswidrig We're using them (actually Liberator does for us), but we want to give more detailed explanations for what went wrong.

@qqq > On the other hand, the off-heap store refers to (serialized) objects that are managed by EHCache, but stored outside the heap (and also not subject to GC). As the off-heap store continues to be managed in memory, it is slightly slower than the on-heap store, but still faster than the disk store.

Specifically: When we test the error checking in our Liberator resources, we would like to know we actually triggered the error we intended to, instead of something unrelated.

http://stackoverflow.com/questions/6091615/difference-between-on-heap-and-off-heap

@noisesmith @ordnungswidrig And in the application UI, that uses the HTTP API, we would like to be able to know what error exactly happened, so we can give more hints to the user.

@urzds probably create your own data format for error details. I’ve seen that before and it worked nicely for more API like applications.

Beware that the error message presented to the user needs to take into account the context of the user interaction, i.e. the current state on the client. A good HTTP api is stateless so the server resource doesn’t have clue about the context of the user interaction.

E.g. we emit 422 Unprocessable Entity when we get send an object that is somewhat broken. Or if the object is inconsistent in itself. Or if it refers to another object that does not exist. Returning just one error code (422) in this case seems like it will lead to very difficult debugging later.

That’s right.

@noisesmith , @dpsutton: cool, thanks!

Shall we move the discussion to #liberator? This is getting into details.

Sure. Thanks for being so patient with me. 🙂

What's the current recommendation for signal handling? I see some libraries but curious to hear what's actually in use

@colliderwriter it's pretty simple to do with interop https://gist.github.com/noisesmith/e6a3e0ae424e129ece19

@noisesmith Thanks, that's an upgrade for me

it does use sun.misc, but I'm not sure there's any way around that for handling signals

I had been using an addShutdownHook snippet but it had poor selectivity. This is much better

Does anyone have a PURE CLOJURE solution to the following problem: FILE-BACKED MAP STORE. So I want the following operations: (create-db "file-name.db") (.put db key value) (.transact! db) (.rollback! db) (.close! db) So it's just a kv store, where k/v = clojure types. I need it to support transact! and rollback!. I also need open/close. Lastly, I want this to be pure clojure. Is there any library out there that provides this?

you want a map zipper

i think

maybe

err not a zipper

i forget what it’s called

but it stores it’s diffs in a list

so you’d have a list like

[{} {:add {:a 1}} {:add {:a 2} {:add {:b 3}} {:remove :a}]

and you’d get

{:b 3} as the output map

wait do you need that to be an actual db or just in memory store?

oh you said file backed

nvm

ignore me sorry

@noisesmith @ordnungswidrig Thanks again for your help!

@octopuscabbage : yeah, I have an in memory version working, it's the file-backed part that I'm stuck on

@qqq just use a proper database, even if the file based map in pure clojure existed, I wouldn't trust it for correctness

especially since I'm not sure what JVM file IO semantics are

we have immutable data structures in memory, we don't have immutable files

it's not clear why algorithms taht work in RAM won't work also work on disk

because your file can be accessed by two vms at once

because a vm can crash in the middle of using the file

a lot of extra issues come up with files

it's all possible to manage, there are programs built to manage these issues, they are called databases

suppose I'm an idiot and bent on reinvengint the wheel for educational purposes

where could I. learn how to write a simple java program that handled all this? (and then I can convert it to clojure)

you could start by looking at sqlite - it's a small codebase

as in, reading their code to see how they designed it

dealing with a filesystem is not in principle different than dealing with network connections.

https://sqljet.com/ there's actually a pure java implementation

@mobileink except for the persistence part

howso?

filesystems break.

One of the problems to solve is to make sure the data is written conistent and persistent to the file system bevor you acknowledge the transaction.

@sveri: yeah, I don't know how to solve that problem

@qqq this is one of the solutions for instance: https://en.wikipedia.org/wiki/Write-ahead_logging

@sveri and there us no way to do that, even in principle, since io is involved. the best we can do is statistical: minimize the probability of error.

Additionally some filesystems pretend they wrote the data, while they did not, but do so a few seconds later.

is there any reason why these can be done in other languages; but have not been done in clojure?

@qqq same reason we don't have a web browser written in clojure - it's a big hard problem and a usable solution represents a lot of work

@mobileink In a pragmatic way, the problem is solved, by using replication, yes, in some very very rare cases, it x different datacenters go down at the same time, it still might fail. But it is possible to make sure the data ist stored persistently

alright, I'm going to study https://github.com/DnAGreenberg/bjitcask/tree/master/src first

interesting, hadn't heard of that thing - I wonder if it works as advertised

the code that I would really love to read -- pure clj implementation of leveldb

@noisesmith it's possible to arrange things such that you have a high degree of confidence that everythin is hunky-dory. it is not possible to guarantee it, as is the case with truly functional programmer and proof engines, etc.

practical programming is inherently probabalistic. (sp?)

we have good consensus algorithms that work on the network, as long as you allow "quorum could not be reached" as a valid state at least - there are some guarantees possible as long as you reify failure

most people don't do it right though, but there's solid theory, and decent implementations out there

but yes, distsys is distsys, whether you mean between two processes on one box, process and os, or over the network, etc.

i don't see how an algorithm could work on a network. fwiw i'm not being intentionally obstreperous, i think this is not only a fascinating topic but one for which we do not yet have widely accepted models.

@mobileink you might find Leslie Lamport's work interesting, RAFT for example

they are notoriously easy to implement incorrectly, but they work

thanks, wiil look (someday 😉 ) my go-to guy is Robin Milner. his book on pi calculus completely blew my computational mind.

yeah, this all follows up on the pi calculus work

@mobileink for a much more intertaining take, there is the jepsen series by aphyr, where various claims about distsys behavior are tested under simulations to expose bugs in widely used databases and messaging systems. Also, jepsen was implemented in clojure https://aphyr.com/tags/jepsen

fwiw, robert soare is worth a look. most all gis stuff is way over my head but he has interesting things to say about the topic - like, it's really all about what's not computable, or ath like that. his magnum opus came out recently: http://www.springer.com/us/book/9783642319327

cool

yeah, check out his essays too at http://www.people.cs.uchicago.edu/~soare/Turing/ really eye-opening.

To those using emacs, I enjoy clojure-align but is there a way to unalign? Ex: collapse all the space between, say, a maps keys and values to a single space?

wow, clojure-align is awesome

ya, use it on lets, {...}s, everything. I love it with paredit-reindent-defun

(`M-q` and M-Q for me, super handy)

original question remains though, can I somehow get a clojure-unalign

if you don't mind mungling some strings, how about s/\s+/ /g it finds adjacent spaces and reduces them to 1 space

I guess you'd have to reindent after that

do you know of any fn, say in paredit, that would let me limit that to the current s-exp, or something?

perhaps paredit-point-at-sexp-start and paredit-point-at-sexp-end ?

you could use that to define a region

I'll see what I can do, thanks!

i think that's enough to roll with!

there goes an afternoon 🙂

right! I'll have to do it prob this evening

is there an analog to haskell's maybe? ie https://www.stackage.org/lts-8.8/hoogle?q=maybe or simply (defn maybe [b fab maybea] (if (nil? a) b (fab a))) (i like sticking to the core fns though)

fnil comes close, but not quite

josh.freckleton I think the way I use some-> and some->> scratches a similar itch while not at all being the same thing

(short circuiting a chain as calls, as soon as one step is nil)

ah yes, these! I'd forgotten about em, thanks!

why doesn't merge use transients for hashmaps? 🙂

It could probably, submit a patch

I have a function like this: ` (defn tmap [obj lst] (reduce (fn [tobj [k v]] (if (nil? v)

@josh.freckleton there was some discussion of that at https://blog.skyliner.io/fourteen-months-with-clojure-beb8b3e4bf00

(defn tmap [obj lst]
  (persistent!
  (reduce
    (fn [tobj [k v]]
      (if (nil? v)
        (dissoc! tobj k)
        (assoc! tobj k v)))
    (transient! obj)
    lst)))

there's http://dev.clojure.org/jira/browse/CLJ-1458

that seems better thought out

If only someone would do some perf tests, I would screen it 😃