if I have a webapp where users uploada pdf and it's converted to jpegs and I process the jpegs; am I better off doing this in clojure in java server side or in js using pdf.js clientside?

after you process the jpegs, do you then store them, validate them somehow, or something else? @qqq

one consideration is transit size -- which is cheaper/easier to send? one pdf, or several jpegs? you would tend to process client side if the post-conversion jpegs would be much smaller to send over the network

@qqq Not sure if this is directly helpful for your project, but just wanted to mention I'm actually working on something similar but with a different approach. Using Clojure as a Code Generator for ImageMagick commands to turn pdfs into pngs.

@joshjones @mattgeb : the XY problem is: I'm writing some custom code for doing OCR in a case where tesseract-OCR doesn't work that well

the input is scanned PDFs, I need to convert it to jpegs before processing

I'm slightly worrieda bout malicious PDFs, but not too worried

if the size of the pdf roughly equals the cumulative size of the jpegs, i'd say go with whichever method is easier to implement robustly (client side conversion + multiple files sent, or server side conversion + one file sent). client side conversion would save network in the event there is an error in conversion. i'm not up on pdf security though, so do your homework on that one for sure

@ag (and (f x) x) except it'll return false instead of nil when not true

@ag nevermind, just read further down, topic exhausted

why doesn't clojure have an dissoc-in, which is (dissoc-in obj path blah) ... and after doing the dissoc, it walks up the path, and kills all the empty maps

http://dev.clojure.org/jira/browse/CLJ-1063

do you know of a good clojure lib that allows maintaining a list of entries that have a ttl (and also a max size)? i could use ttl-cache-factory but the expiration only kicks in on a write to the cache. i need the expired entries to not be present when i enumerate the entries in the list (or map). of course i could hack around it by making a faux write each time i enumerate the entries. but any better ideas anyone?

there is a clj wrapper for java expiringmap lib https://github.com/luminus-framework/expiring-map

@quan thanks! does it allow specifying a max size? or is that via using interop on the underlying java object?

i don't think clj wrapper lib has, but original java lib support it https://github.com/jhalterman/expiringmap

anw, wrapper code is simple & you can customize for your need https://github.com/luminus-framework/expiring-map/blob/master/src/expiring_map/core.clj

Is there a particular protocol to use when defining resources to be used with with-open ?

Also, is there anything else that resemble Python contextmanager in Clojure?

@lsenta java.lang.AutoCloseable or any .close method really (`java.io.Closeable` works as well)

Thanks Thheller, I still miss a detail on how to use these,

I have a function (make-something) and (clear-something X)

uhm they are java interfaces not protocols

(defrecord Something []
  java.lang.AutoCloseable
  (close [this] ...))

or deftype depending on what make-something creates

Yea that's where I'm not sure what's the clojure way to do it, I use a library that gives me a "store", I don't particularly want to dig into their implementation details

All I want to do is to have a way to make sure the store is closed when I'm done with it

coming from Python my reflex is to build something that'd look like

with my_store() as s:
  do something

hmm if you don't control the store it will be hard to attach the close to it

(let [x (make-something)] (try (do-with-something x) (finally (clear-something x))) maybe

That's more or less the with-open macro 😅

I personally don't like cleanup in finally but it ensures the clear-something is called

yeah I know 🙂

the problem usually is an exception in close .. usually nothing you can do about that but still

with-open is built with a .close method in mind which is a java idiom which most java things implement

So my only choice is to write a macro, every time I have a (start) (stop x) to call

some clojure things as well but if the lib you are using doesn't you are out of luck

well you don't really need a macro for that but yes every different impl requires a new solution

https://github.com/stuartsierra/component and others try to solve this issue by introducing a common protocol

which I dislike personally due to the invasive nature of the protocol but still

Is anyone using clojure for system programming? Maybe with a specialized runtime?

@thheller I haven't thought of going straight to these, that could help, thanks.

@thheller How is the Lifecycle protocol invasive? Perhaps you will like Integrant? https://github.com/weavejester/integrant

@yonatanel IMHO a "thing" (component, service, whatever you want to call it) should not be coupled to the container it is running in

which you do if you have the component implement a thing of the container

not that component does that as you can implement the protocol independently but some things do

so if a lib depends on component that is a warning sign for me

not sure about integrant ... never used it. not sure I like the multi-method approach

@thheller A more data-oriented approach would be to tell the lib which function to run using a symbol. Is that what you mean?

I have this as a tiny little helper utility, basically the same integrant does but in code not data

{:something
 {:depends-on [:something-else]
  :start something/start
  :stop something/stop}}

something/start will be called as (something/start instance-of-something-else) and (something/stop whatever-start-returned)

really not better or worse than integrant or component, solves the same problem with different tradeoffs

you can't really make this completely data oriented IMHO, so whether you organize things into multimethods or a protocol or a clojure map is up to personal preference really

@thheller There’s also (vaguely similar to your idea) https://github.com/plumatic/plumbing#graph-the-functional-swiss-army-knife and (forgive my shameless plug) https://github.com/kumarshantanu/dime — I’d love to know any other ideas to solve this kind of problem

@kumarshantanu yeah there are at least 5 other solutions I know of and probably 50 other in Java. I used to work on PicoContainer in Java 10+ years ago and still have no idea what the best approach is. Something about the simple declarative nature of clojure maps appeals to me as everything is in one place but other solutions (like yours) attempt to also free you from passing arguments to functions. I don't mind that part so my solution doesn't cover it at all.

Hi all, is anyone aware of a lein tool to clean up dead code ? I only found https://github.com/venantius/yagni but I have some issues with it. It also seems a bit outdated

@aspra - Have you looked at eastwood? It does a lot more than yagni, but it does also check for unused code.

@jstew I just saw that it has linters for it. Thanks a lot!

I do also use yagni with boot-check and it works fine, even though it's dependencies are older. I haven't had any issues with it, really.

I suppose it's one of the nice things about boot, running things with older deps in pods, where it won't affect the rest of the system.

I have never heard or used boot before. Thanks for the tip

@jstew quick update: eastwood doesn’t seem to do what I want. A lot of nice stuff but not unused defns

Ah, unused locals only? That's what I was afraid of.

yeah. Thats an interesting approach I suppose http://diogo149.github.io/2014/08/30/dead-code-clojure/

@jstew > I do also use yagni with boot-check and it works fine, even though it's dependencies are older boot-check uses latest releases:

venantius/yagni "0.1.4"
jonase/kibit "0.1.3"
jonase/eastwood "0.2.3"
lein-bikeshed "0.4.1"

@tolitius - yagni's dependencies are out of date (according to github anyway), not boot-check's. boot-check has always been a pleasure to use.

@jstew ah.. got it. thanks for the feedback 🙂

@jstew this works https://gist.github.com/joelittlejohn/4729776. Maybe quick and dirty but does the job

(yagni author) I’d welcome PRs to update deps, etc 😉

@aspra Quick and dirty, yes... Looks like it will work though. Would have to be modified for cljc and cljs is the only immediate issue I would have with it.

@venantius thanks! In my case I have some dependency issues it seems. I think I have more of an issue in my project that with yagni. Just saw that it was not updated for years and it assumed it is not maintained. Sorry 🙂 If I find some issue in particular I will let you know

Semi related, many clojure libraries look "abandoned" but are really just mature and feature complete. Clojure and most of it's 3rd party libraries have great backward-compatibility which eliminates the need to keep updating code when clojure or a dependency is updated.

true, good point. But you know how it goes: it is not my code so must be the library 🙂

@jstew what about clojure makes that true?

Hello! Is there such a thing as a lazy classloader? Something that downloads a jar and puts it on the classpath at the last minute?

(You know, from maven central or something.)

I appreciate that I could, uh, ship a zip file and maybe run lein deps on it

Context: lambda.

https://github.com/rkneufeld/lein-try ?

octopuscabbage huh, good idea; I was considering lein in fact

more context: I’m running lambda, there’s a 50MB jar limit, they don’t ship Java AWS SDKs, all java AWS SDKs together is >50MB

so I was hoping I can do it from the JVM maybe

we dynamically load clojure files

but i’m not famiilar with how to do it with jars

@jstew However solid test coverage isn't always available. In that case difficult for a developer to confirm compatibility without writing their own tests (ideally sending a PR to the library) or more often the case taking a leap of faith. I had exactly this kind of question this week, finally asked the library maintainer because there was no other way to know a priori. And the library is complex enough that I'm in no position to write tests for it. The maintainer replied "it should work, we're making it compatible with newer versions of [that dependency] without modifying the code".

something here about a hack to load jars onto classpath https://groups.google.com/forum/#!topic/clojure/AJXqbpGMQw4

clojure.core/add-classpath https://clojuredocs.org/clojure.core/add-classpath

deprecated

but it uses

URLClassLoader.addURL internally it seems

use an http client to download the jar to some cache and then use one of those methods to load it

@octopuscabbage @gonewest818 - That's very true. Test coverage is the only way to prove compatibility, and often poor tests are created. I speak from my own experiences compared to say, Ruby and node.js.

@octopuscabbage yep, looks good. Thanks!

@octopuscabbage https://github.com/zcaudate/lucidity provides something that does this. http://docs.caudate.me/lucidity/lucid-package.html#entry__lucid_package__pull

If anyone is interested in going back to IRC but found client tools to be not as good as Slack: I recently found an app called Riot. It integrates a lot of different chat protocols into one experience and it actually succeeds in being a great user experience, IMO.

I've been following Clojure IRC again because of it and it's more active than last I checked.

Any follow-up on the Slack vs X debate should happen in #community-development -- not in this channel /admin-hat-on

*.cljc is still considered a GOOD THING right? One particularly frustrating thing is having to use conditionals for cljs.reader/read-string vs clojure.core/read-string Is there any plans of creating a "standard cljc library" where a the conditionals are hidden away, and it exports a common set of functions useable in both clojure and cljs ?

i think i recall reading somewhere that the way clojure's lazy sequences work is that they are realized ahead of time in small batches (like 32 elements at a time)

is that sentiment correct?

Clojure West videos out already, awesome!

if so, is there a way to ensure that elements are only figured as late as possible?

@bcbradley would https://clojuredocs.org/clojure.core/delay be an option for what you're trying to do?

@drewverlee where I can find em?

https://www.youtube.com/channel/UCaLlzGqiPE2QRj6sSOawJRg

maybe it would help if i explained what i'm trying to do

@drewverlee thanks

i don't think delay is what he wants. ie, (take 2 (map get-id-from-database [1 2 3 4 5])) would ideally be only 2 database calls. but lazy in clojure doesn't necessarily work like that if i understand correctly

i've been working for some time on a game engine in clojure and i'm really hoping to elevate the model above the conventional way game engines are traditionally built

i want to treat the sequence of game states as an infinite lazy sequence

something like "unchunk" then? from here: http://stackoverflow.com/questions/3407876/how-do-i-avoid-clojures-chunking-behavior-for-lazy-seqs-that-i-want-to-short-ci

thanks thats probably what i wanted!

i need to find some resources that really explain how clojure chunks its lazy seqs somewhere

It's funny, I was just reading about chunking and creating lazy seqs in joy of clojure on the bus this morning.

what does chunking have to do with lazy seqs?

to my knowledge a lazy sequence in clojure isn't necessarily as lazy as possible

many operations which return a lazy seq and take a lazy seq will realize more elements than necessary (buffering up)

this is presumably because doing so is more computationally efficient for the most common cases where the elements are very small

its undesirable at times though

I think it depends on the seq you're operating on

does it?

i didn't know that

how does it know the difference between something that is expensive versus something that is cheap?

it depends on the producer

to take fogus' example:

(def gimme #(do (print \.) %))

#'user/gimme
user=> user=> (take 1 (map gimme (range 32)))
(................................0)
user=> (first (map gimme (range 32)))
................................0
user=> (first (map gimme (iterate inc 0)))
.0

i would have expected the last example to output a single .

it prints the . and val

if a lazy sequence is built up via lazy-seq then it is not chunked

alright thankyou for your help! thats what i wanted to hear!

(chunked-seq? (range 100)) and (chunked-seq? (iterate inc 0)) produces true and false respectively.

range is special for performance reasons IIRC

I suppose if you're in doubt you could always check with chunked-seq?

https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj#L3168

partition uses lazy-seq internally so this will work:

user=> (first (map gimme (partition 2 [1 2 3 4 5 6 7 8])))
.(1 2)

tell me if you think this approach is sound: (iterate (partial time-travel n) [initial-events initial-states]) => (target-events target-states)

time-travel is a function that takes an int and a vector of [events states]

if n is 1, it just marches the game state forward one frame, if it is -1 it rewinds

you could do integers other than 1 and -1 of course

i'm still experimenting so idk if my ideas are sound yet; if you've got any advice i welcome it

as long as you don't hold onto the head then I think you're fine with that approach

i want to provide different kinds of time-travel

time traveling without rewinding the events (so mouse movements and such don't get replayed)

and full time-traveling, rewinding events and all, so the game replays itself

you’d have to be careful with side effects

yeah i'm hoping that i can rely on clojure's default immutability to give me a solid base of operations to work with so i don't have to use side effects

I would probably store the result of the transitioned state in a vector

that way it is indexed and doesn't include side effects

i was thinking about that too

it would make rewinding much easier too i think

i wouldn't have to make each operation invertible

I know that Compojure exists. I'm trying to learn Ring by doing basic stuff by hand. I'm looking at https://github.com/ring-clojure/ring/wiki . Where is the section for handling GET/POST requests? (I really want to do this by hand to learn how Ring works; please don't sugest a framework 🙂 ).

that's what I did for a snake game

store the frame offset and make each frame available in an indexed coll

dec the frame offset for rewinding

hrm, that seems like a solid approach

if the state is stored in a vector, and the next state is always conj on the end of it

i'd extend that approach to events in a similar way-- there would be an event vector

that way I can time travel on one, the other, or both

that makes macros trivial to implement, makes prince of persia type time travel easy, and it also makes "real" replays easy too

you’re going to have to use side effects eventually, you can usually facade them into being idempotent though

for example writing stuff to disk

i was hoping to just memory map the game state to the disk

and avoid that altogether

that’s just an example, you should figure out a of a strategy now that fits in your framework

yeah i'm just trying to consider this from a fresh point of view

i know how i would do it in c, or java

but clojure gives me a lot of powerful tools i don't have in those languages

i'm trying to think of how to leverage them

Can you please recommend libs/tools to build SOAP server from scratch on clojure. I’m looking to clj-soap, but seems like it no longer maintained.

why is clojure.lang.PersistentQueue not a part of the core library?

it is

where is the documentation?

it's a data structure

works like a seq. what documentation do you desire?

idk, there are some set operations that aren't supported by any other structure, like union and intersection

there are some map operations that aren't supported by other ops

think of a queue as a list

vectors have efficient nth behavior

https://clojuredocs.org/clojure.core/pop

you can conj and pop but no random access

ok, so no efficient nth

this is what i desire in documentation

there are all kinds of ways to implement a queue, it'd be nice to have an explanation

https://en.wikipedia.org/wiki/Queue_(abstract_data_type)

c++'s deque DOES support random access for instance

and it satisfies the interface for a queue

i can understand not having a reader literal syntax for every data structure out there

but queues seem pretty basic

#[a b c] would be something i would expect

#queue is a supported tag in transit IIRC

I was expecting a reader literal as well. Not a deal breaker for me though, it works as a Queue, like I expected it to.

hmm nvm that's a data tag for cljs

why isn't there a clojure wrapper function for it?

like hash-map for instance

it feels like it is "there" but "discouraged"

honestly, I do not know. I've seen it asked on the google group and stackoverflow and I don't recall any sort of definitive answer.

i can't think of any other reason to keep queues hidden like this

what is the queue use case?

conj to the end and pop the front?

yeah

i can't keep every game state alive, so i have to "forget" some of them

seems like an ideal use of queues

I think you want a vector for that

isn't popping from the front of a vector slow?

or should i do subvec?

subvec

hrm ok, that works for me

will it forget the head and all that?

not if you have a reference to the head

the game state should just contain the vector value

after thinking about it a bit this is basically what i have:

(future time) => more-time

(past time) => less-time

(present time) => state

time, more-time, and less-time are just vectors like you describe

present basically gets the last element of the vector

future marches the game engine to the next state, so its expensive

past just pops the last element off

the main game loop is defined as (iterate future initial-time)

the way i see it every game is just a succession of states

vectors aren't lazy so they don't have the head reference issue

even games that give the illusion of time-travel are really still just a succession of states

it just so happens that some of those states look like prior states (the time-traveling bit)

in other words, if you view (future time) as (time-travel 1 time) then "time travel" in games like prince of persia are actually time-travel WITHIN time-travel

atleast thats what i've come up with

@jr thanks for the advice, it helps

np good luck with your game engine

thanks!

does anyone have a minimal example of: restful service client = cljs + cljs-ajax + transit server = clj + ring + transit I've played with ring; what I don't get is "how do I return an clj object in json/transit encoded" ?

you have to read the transit data by detecting the content-type header

found it: https://github.com/swannodette/transit-example/blob/master/src/clj/transit_example/server.clj

@jr: sorry, I don't understand what you mean

use something like https://github.com/ngrunwald/ring-middleware-format

for the server

yeah, I have ring-middleware-format installed as adependency

the problem is, there's no sample code on that page

I don't know how ot put those pieces togehter

I think https://github.com/swannodette/transit-example/blob/master/src/clj/transit_example/server.clj#L77 is all I need

copying/pasting taht should make it work

don't copy and paste

do you understand ring middleware?

wrap-restful-response :formats [:transit-json] is probably the middleware you want

it will encode your :body response as transit-json and add the header Content-Type: application/transit+json

you will then need to decode that transit response on the client to get cljs data

cljs-ajax does that automatically and uses transit as default

@jr: I generally prefer to (1) get something working first, then (2) try playing with working example + reading docs

plain doc reading == I can't see how all the pieces fit together

on client side, I do cljs-ajax/POST ... on the server side, I get a ring request, the :body field has an object of type org.eclipse.jetty.server.HttpInputOverHTTP how do I extract the clojure data of the POST from this HttpInputOverHTTP ?

okay, got it all working, so server side is: #(transit/read (transit/reader ... :json))