Fork me on GitHub
#clojure
<
2021-12-20
>
lilactown02:12:05

I'm down a bit of a rabbit hole

lilactown02:12:51

is there a library for doing seq ops, but continuation passing style? i.e.

(reduce
 (fn [k acc x] (k (+ acc n)))
 0
 [1 2 3])
;; => 6

hiredman04:12:21

(reduce (fn [f x] (fn [k] (f (fn [acc] (k (+ acc x)))))) #(% 0) [1 2 3])

hiredman04:12:43

If you look at usages of fold in haskell, you often need to watch out for space leaks, because the accumulator becomes a big deferred calculation, instead of, you know just a number

lilactown05:12:36

how does trampolining behave? I don't yet have an intuition for how trampolined algorithms act w.r.t. memory

lilactown05:12:25

I was asking the question specifically in the context of writing a bunch of algorithms using CPS + trampoline in Clojure, and looking to see if maybe someone had done it for me

Ben Sless06:12:40

If you mix it with trampoline I don't think at any point you'll allocate more memory than the original collection. You can map over a collection and create all the continuations, then compose them and trigger them with trampoline

hiredman07:12:56

The thing to avoid is mixing and matching, my use of clojure.core/reduce above could be altered to build a fn that could be handed to trampoline to run the reduce, but because it is split like that, reduce building thinks, trampoline discharging at the end, stuff hangs around in memory

hiredman08:12:45

A proper cps reduce for trampoline would use trampoline to drive the loop, so thunks are discharged immediately

Ben Sless08:12:51

The loop being iterating over the collection? i.e. have to reimplement reduce

emccue04:12:14

This might be a dumb question, but now that cognitect has been acquired by nubank do any of its “economic priorities” change? Like, facebook’s primary revenue stream is ads so making rocksdb open source was by whatever calculus the right call. I don’t know if that is community support or PR or what. There is no proven, well supported open source direct alternative to Datomic and i’m just hoping out loud. (crux is, to the best of my knowledge, a different kind beast)

cddr07:12:02

Suspect their need to get people with little/no knowledge of clojure "up to speed" as quickly as possible has increased.

winsome14:12:11

Have you seen https://flur.ee/? It's an immutable 3+ tuple store written in clojure. It's open source and has commercial support.

emccue14:12:15

> The Web3 Data Platform What a poisonous description

emccue14:12:55

1. No internal application has need for “the blockchain” 2. That has only existed for 2 years 3. If you work on it i’m definitely being rude, but i’m okay with that. I am absolutely sick of “web 3" crap

winsome14:12:39

It's not actually a blockchain in the cryptocurrency sense, just in the git sense - a "chain" of "blocks" that you can traverse to verify provenance.

emccue14:12:16

Then their website is straight lying

emccue14:12:56

> With blockchain cryptography, optional decentralization, and configurable rules written directly to the ledger,

emccue14:12:07

am i going to be running POW on my servers now?

winsome14:12:17

From the technical docs: > Fluree is a private blockchain: Fluree's network and consensus will be more like Hyperledger Fabric than public blockchains like Bitcoin and Ethereum. This in no way means the data housed in Fluree cannot be made publicly available - it can and often is - this distinction is only focused on who gets to vote on the validity of a transaction. Public/Private is a fairly big tradeoff, offering different benefits - while not common, Fluree has plenty of customers that utilize both in the context of a single solution.

winsome14:12:38

There is no mining

emccue15:12:39

i’ll take another look after my initial disgust dies down

👍 1
enn21:12:06

To speak to the original question, I imagine that the economic incentive to keep Datomic maintained and operational is stronger than ever (since it is used by Nubank). On the other hand I imagine that Datomic licensing revenue is a drop in the bucket compared to Nubank's other revenue streams. So I imagine that the incentive to add or maintain features that are not of interest to Nubank has been reduced. I have no idea what (if any) features those might be.

Joshua Suskalo05:12:34

datahike is probably the closest to a direct alternative

👍 1
Ben Sless06:12:17

Is there any material around or a good example for an HTTP API client written in Clojure? Trying to keep it data oriented

emccue06:12:44

Do you mean 1. A client library for an http service 2. A library which helps you make http requests

Ben Sless06:12:29

A client library for an HTTP service, for http requests I have plenty of options.

Ben Sless06:12:27

I want to build a data driven client with the terminology and concepts of the API I'm communicating with. Base URL and authentication are handled by the client, actions I want to model and drive with data

emccue06:12:32

dumb answer is it depends on your api

emccue06:12:10

(client/send client (client/some-op {:arg-1 123}))

emccue06:12:39

like, if you want a generic “send” you can make functions to create your api payloads

Ben Sless06:12:48

It always depends. Any good reading material or references you'd recommend? I'm displeased with my first iteration

Ben Sless06:12:22

Okay, the question is how much of the op should be tied to the client

emccue06:12:30

not really. Remote api libraries can often just look like local libraries from a contract perspective

emccue06:12:45

(obv. they aren’t otherwise we would all love RPC)

Ben Sless06:12:34

I can do everything with types and protocols, but that kinda sucks from a Clojure perspective

emccue06:12:37

so maybe a first step if you want to provide this abstraction layer is to doodle out what the available operations are, what they options they accept, and how if at all they can be combined

emccue06:12:57

can you give like 3 example calls?

Ben Sless06:12:29

Not really 🙂

emccue06:12:53

so one way is just some.resource.api with create, update, delete, some-command, other-command if its a REST style api

emccue06:12:08

if it is more complicated for whatever reason - command pattern

Ben Sless06:12:10

The doodling bit is why I end up in rewrite land. I started in some direction, got a feel for it, then decided there's a better way

emccue06:12:35

{::api/op :do-thing
 :arg-1 123}

emccue06:12:53

define all your ops in a schema, etc

emccue06:12:06

this is (kinda) how cognitect-aws-api does its thing

Ben Sless06:12:11

I think that's where I'm headed

Ben Sless06:12:28

That could be a handy reference, thanks 🙂

Ben Sless06:12:40

The API itself has some primitive operations, then all the interesting application level operations are done with them. I think this is good enough to get started

Ben Sless06:12:23

Another thing, would you rather use an api where the arguments are provided as a map or as positional arguments?

Ben Sless06:12:59

In particular for the data driven part

emccue06:12:17

i do like cognitect aws api’s ability to list the operations you can perform

emccue06:12:47

and the fact that the maps you pass are 1-1 with what the api takes, not kebab-cased for vanity’s sake or whatever

Ben Sless06:12:16

Luckily my API is cultured enough to use kebab case

Ben Sless06:12:51

Maybe it will be external in the future though, it still takes kebab case

Ben Sless06:12:22

I'm still a bit on the fence regarding maps vs. positional

Ben Sless07:12:10

The client itself should take a map, that makes sense, but if I want to provide convenient functions to build that map, do I go for positional or map args?

Ben Sless07:12:59

(api/call client op-map) 👍 Building the op map :thinking_face:

Ben Sless07:12:47

;;; Positional

(defn prim-op [x y z w])
(defn op1 [a b] (prim-op a b "foo" "bar"))

;;; Map

(defn prim-op [{:keys [x y z w]}])
(defn op1 [{:keys [a b]}] (prim-op {:x a :y b :z "foo" :w "bar"}))

Ben Sless07:12:55

Who do you like more?

Ben Sless07:12:37

ofc I could support both with varargs

hiredman07:12:36

If you haven't looked at cognitects aws API client, I think that is pretty great

hiredman07:12:17

A single invoke function that takes a client instance and a map that describes the operation to perform

hiredman07:12:55

Along with a doc function to get docs on the operations

hiredman07:12:03

Ah I see that is basically the call stuff above and you are now just talking about helpers to build the op map

hiredman07:12:41

I wouldn't bother, clojure has great functions already for building and manipulating maps

hiredman07:12:21

But I would try to wrap the API behind a single function as much as possible, describe operations as data, have a single function to execute them

hiredman07:12:50

Make the single function a protocol extendable via metadata for easy testing

Ben Sless07:12:46

Thank you both, I feel better oriented

Ben Sless09:12:22

looking in the aws-api, where can I find the specs for the different requests?

jumar12:12:42

This might be better suited for #aws

dgr16:12:18

Are you thinking of using the Swagger metadata to allow you to auto-generate the programmatic interface on the client side?

Ben Sless16:12:55

If I had any, I would have

hiredman16:12:25

I guess it depends on what you mean by specs for different requests, the library provides a function for listing operations and for getting docs about them

hiredman16:12:12

I know it runs off of the json docs that describe aws operations, and I vaguely recall maybe those being translated to clojure.specs but I am not sure

hiredman16:12:58

Html documentation is often regular enough in format that you can parse it and generate a client from it

hiredman16:12:24

I've done that a few times, one time the API we were integrating with obviously generated their docs from swagger, but didn't publish the json for it

rickmoynihan11:12:10

Is there an argument for updating the docs in clojure.instant to point people towards considering java.time instead (and possibly deprecating it — though there are a lot of subtleties here)? A colleague unfamiliar with the history of java.util.Date and the various java time APIs had started using read-instant-date; then later discovered that java.util.Date doesn’t handle timezones properly etc.

1
vemv13:12:07

but even in modern Java, an Instant does not equal a ZonedDateTime, they're different classes so clojure.instant might make sense when considered for representing "instants", and when not having extra expectations about what it provides I could be wrong though :)

rickmoynihan13:12:07

Yes, these are the sorts of subtleties to which I was referring… i.e. java.util.Date’s constructor is not actually deprecated; just almost all of its methods etc…

👀 1
vemv13:12:36

the Eastwood linter observes deprecated Java methods, so one could use it as a hard CI check I have it on the radar to allow users to specify extra methods that they'd want to consider "deprecated" even if they're not. So one could be extra picky around Date.

rickmoynihan13:12:44

Cool good to know. I guess I’m just saying — 1. It might be useful to add some notes in the docs saying “you probably don’t want this, look at java.time instead”. 2. Potentially asking the powers that be to consider whether the ns or some of the functions within it should be deprecated or not.

vemv13:12:40

Personally I would be skeptical about deprecations, that's simply my impression though. An alternative path could be further developing what an #inst means, other than a cool reader representation I'm not sure there's exactly a vast API supporting it (such that, for example, one could not use a 3rd-party lib at all)

rickmoynihan13:12:19

I definitely agree deprecation might not be the right thing — that’s why I’m just suggesting people more knowledgeable about the subject than me might want to consider it; rather than advocating for it to actually be deprecated. I’m not sure I see a reason to pursue your alternative suggestion though. We have clj-time already, and tick etc which are both excellent libraries and provide more comprehensive support for time.

rickmoynihan14:12:17

(I meant clojure java-time not clj-time, though that is ok for legacy joda stuff)

vemv14:12:46

It would be quite the precedent for Clojure to recommend a specific third-party library :)

rickmoynihan16:12:35

😆 True but I’m not suggesting they recommend a clj library; I’m suggesting the docs point people to java.time which is part of the JDK since Java 8.

dgr16:12:31

java.time is the right answer. This is one of those times where the Java underpinnings of Clojure start to bleed through. I don’t know that there is a great answer here. The world of dates and times is a notorious mess in most languages. Anybody who is writing any code that deals with dates or times needs to understand the mess and navigate it carefully. In other words, this is bigger than simply avoiding instants in favor of something else. You need to know what else you would use and how to convert between multiple historical attempts at getting it dates/times correct.

vemv16:12:53

ah, java.time the package I had trouble parsing that 😇 one particular thing I recall is a difference in time resolution between JDK8 and later versions

rickmoynihan10:12:33

@U7BEY9U10: I agree. My point is really that clojure.instant isn’t really helping language users, such as my colleague with that complexity, but arguably adding to it. I guess it might be useful to add a function read-java-time-instant to that ns; and add a docstring to encourage users to do (binding [*data-readers* {'inst read-java-time-instant}] ,,,)` I’m aware that we can’t obviously change that default, and that (.toInstant j-u-date) is all a user essentially wants to do most of the time (or perhaps just use java.time directly), but it would be nice to encourage them to fall into the pit of success here rather than one of misery and despair :-)

tami516:12:59

is there a cleaner way to write this :)

(fn [opts _]
  (fn [handler]
    (fn [request]
      (wrap-initialize-request handler request opts))))

Ben Sless16:12:27

Besides a macro, not much. Why?

Joshua Suskalo16:12:46

You could make a function that takes a function of three arguments, the opts, handler, and request, and turn it into this nested structure.

(defn make-handler
  [f]
  (fn [opts _]
    (fn [handler]
      (fn [request]
        (f opts handler request)))))

(make-handler
  (fn [opts handler request]
    (wrap-initialize-request handler request opts)))

dgr16:12:23

That’s actually pretty clean and shows what it happening. If you’re thinking there’s a lot of boilerplate there, then yes, you could use a macro or helper function to reduce it.

tami516:12:25

@U5NCUG8NR you know what I believe that exactly what I'm looking for, Thank you. Yes, a macro seems a viable solution too

👍 1
qqq17:12:22

Apologies if this has already been discussed to death; for creating a new from scratch Clojure project, what is the best way to ensure no dependency uses log4j ?

emccue17:12:16

clj -Stree | grep 'log4j'

Alex Miller (Clojure team)17:12:39

I would actually recommend clj -X:deps list | grep 'log4j' - the former would list it even if excluded (with an X at the beginning)

Alex Miller (Clojure team)17:12:27

but note that that does not protect you from a lib (wrongly) including log4j directly in its own jar

Joshua Suskalo17:12:58

The above is a great way to see if a dependency depends on log4j if you're using deps.edn. From there you can put exclusions on any root-level dependency that depends on log4j to ensure it's not included (and put in e.g. a log4j to slf4j bridge as a replacement, and use that bridge and a different logging implementation like logback that doesn't suffer from the vulnerability). You can find similar deps listings and exclusion methods for other build tools.

👍 1
qqq20:12:04

As a followup to the above, is there a way to run a Java program with the option of limit UDP/TCP to 192.168.x.x ? I.e. this is a Clojure program that takes data from filesystem / other machines on private network, and it has no business talking to general internet.

p-himik20:12:56

Wouldn't it be better solved by the thing that manages that app, e.g. its container? Even if it runs on a raw Linux host, you should still be able to limit IP ranges the app can use. That's probably also true for non-Linux hosts.

valerauko21:12:04

it'd be safer to do it at a lower level (container, os) so that if something does escape the app (shell call etc) it still gets limited

jumar12:12:30

Exactly. A long time ago we used SELinux for this kind of stuff - I'm not sure if it's still a hot thing but it's doable that way. Or you can isolate machines instead via things like AWS security groups.

mal00:12:44

firewalld on some linux’s. we basically use a block zone that blocks everything but then you allow ip’s to that.

diego.videco21:12:04

Why does this function return nil ?

(def sm (sorted-map-by
         (fn [_ _] 1)
         "item1" 1
         "item2" 2
         "item3" 3))
(sm "item1")

diego.videco21:12:28

I want to keep my keys manually sorted

hiredman21:12:03

that isn't how sorted maps work

hiredman21:12:25

sorted maps compare keys using a comparator and sort things that way https://clojure.org/guides/comparators

hiredman21:12:41

it sounds like what you want is commonly referred to as an "ordered" map, meaning it keeps keys in order of insertion, which there some libraries that provide that kind of thing, but nothing in clojure itself

hiredman21:12:16

in particular your function that always returns 1 means keys never compare equal, so you can never look anything up

diego.videco21:12:44

would need something stateful like a counter right? Guess I better use a vector and use into when I need a map

hiredman21:12:59

an ordered map is a fundamental different thing from a sorted map, and you cannot build one from a sorted-map using a different comparator

colinkahn23:12:15

I think Lacinia uses this library for its ordered maps, perhaps worth a look: https://github.com/clj-commons/ordered