Fork me on GitHub
#clojure
<
2021-11-22
>
Jack Park01:11:05

A small conundrum using deftype: I created a protocol IMyType in file, say, api.clj; I then implement that as a deftype MyType in a file, say impl.clj and now I want to use that type and its APi in another file, say core.clj. How do I go about requiring that type in core.clj, something like [myns.impl.MyType :as foo] (which does not work? This is being confusing since the bulk of documentation, including books, all want to show how in lein, which doesn't deal with requiring from files the way files do.

emccue02:11:45

there is a function you can require that acts as a “default” constructor for MyType called ->MyType

emccue02:11:30

(:require [myns.impl :as myns-impl])

(myns-impl/->MyImpl 1 2 3)

emccue02:11:26

but that kinda begs the question - what are you using deftype for?

Jack Park02:11:36

I did see that when I typed the namespace. Ok. but what I want to do with myns-impl/->MyImpl is use its functions, e.g. store and fetch.

emccue02:11:47

(:require [myns.impl :as myns-impl]
          [myns :as myns])

(let [obj (myns-impl/->MyImpl 1 2 3)]
  (println (myns/store obj 123 456))
  (println (myns/fetch obj 123)))

Jack Park02:11:37

Ah, the canonical question: why? I'll admit, it's a hangover from being a very long-time "plugin widget" guy, where the idea is to have a particular API, and then make it available to users without any code change but against completely different context.For instance, one is as a direct postgres driver, and one as an http interface to something remote.

Jack Park02:11:08

Clojure may not let me do that. I don't know.

Jack Park02:11:54

In Java, it's called "design by interface" - spend all your time working out the APIs, then go implement them as needed.

emccue02:11:56

that explains the protocol, but there are a few steps before deftype is what you want

Jack Park02:11:20

I'm listening :face_with_cowboy_hat:

emccue02:11:30

there is a decision chart somewhere, one sec

emccue02:11:24

cant seem to find it

1
emccue02:11:23

but basically usually what you want is some way to do dispatch

emccue02:11:47

if you just want to dispatch on the first argument - “single dispatch” then a protocol is a good tool

emccue02:11:29

and the usual ways you can provide an implementation of a protocol are reify, defrecord and deftype

emccue02:11:02

deftype is most common if the thing you are doing has some internal mutable state and there is a benefit to having a named type

emccue02:11:40

which is an uncommon situation, hence me asking

Jack Park02:11:44

I'm growing suspicious that my difficulties are more those of learning a new ontology than, necessarily, coding conventions. In Java, I see an implemented APi as a type; clojure doesn't let me just say "public class foo implements bar" so it feels somewhat logical to just say (deftype foo [] bar ...)

emccue02:11:40

can you describe more specifically what you are working on? maybe that will help me give better advice

emccue02:11:44

because you are right about the parallel - that is how you get 1-1 with what you would do in java

Jack Park02:11:04

Sure. I'm migrating a topic map from Java to Clojure, doing a simple version first, but using the simple version to test the needs of a full-monty system later. The first version is to use Postgress driven by Datahike for a Datalog backside; that entails a specific, but simple API, one which can do puts, gets, finds, and deletes. However, there is an immediate need to graduate from the Postgres/Datahike backside to one which is, essentially based on a remote Kafka-based society of agents which, ultimately, does result in a Datalog backside but only after mountains of processing. it's all open source and going up at github. But, I don't want to have to change the internal logic of the topic map system itself when the backside changes. Thus, the design by interface.

Jack Park02:11:51

The project itself is known as OpenSherlock, which fell out of my PhD research which needed the benefits of topic mapping.

Jack Park02:11:44

Right now, OpenSherlock is a Java platform which uses the Python NLP platform spaCy for some of the work. All postgres based. That needs to change.

Jack Park02:11:10

Thank you for asking.

Jack Park02:11:54

Just in case: a topic map is a kind of "knowledge graph" first invented in 1991 in SGML, then transliterated to XML; mine are mostly JSON; a topic map is like a "concept map" which is nodes with labeled arcs, except that in a topic map, the labeled arcs are actually first-class nodes -- relations are as important as the actors they connect.

emccue02:11:02

and so this protocol you are writing is for the backend data source of it

emccue02:11:22

your operations are actually store/fetch or store/fetch-like

Jack Park02:11:50

Yes. They are very generic, "map-like" which can take on radically different semantics depending on the implementation.

hiredman02:11:40

The protocol and the functions it defines are the interface, and they are a group of functions, whose names are namespaced, so when you refer to them by name you usually require the namespace

emccue02:11:01

okay so your current approach makes sense, with the caveat that It probably doesn’t make sense to expose the deftypes as part of the public api

hiredman02:11:37

Yeah, like in Java you might prefer some kind of factory or builder

emccue02:11:54

a direct parallel can maybe be drawn to xtdb, which has multiple backends

👍 1
hiredman02:11:11

In clojure you would have a function that returns something that satisfies the protocol

emccue02:11:20

there is a protocol for a “kv store”

hiredman02:11:38

(where a factory might be some static method, which is very similar to defn in many ways)

emccue02:11:17

but each namespace exposes a ->kv-store constructor function which is intended to be used instead of the default generated ->RocksKv and map->RocksKv functions, which lets it do extra logic besides just setting the initial value of fields

emccue02:11:33

generally speaking those are treated by the community as internal details

emccue02:11:59

you will also notice that the different implementations are defrecords and not deftypes

emccue02:11:28

there are pros and cons to that - technically speaking when you are a record you gain all the semantics of a map. all your fields can be inspected and swapped directly, your hashcode/equals is based on your fields, and you get a toString

emccue03:11:54

but in this context culturally its a known thing to not muck with the fields directly unless you “know what you are doing”

emccue03:11:09

so its a judgement call

emccue03:11:34

there are also more “encapsulat-ey” things you can do like make the protocol be the public api for extension, but not the public api for calling.

(defprotocol IStore
  (store* [this data]))

(defn store [this data]
  (store* this data))

emccue03:11:34

you see this in the apis for clojurescript - and its something you do if you want maximum flexibility later on with external implementors, but is often overkill

emccue03:11:17

(like if you wanted to introduce another dispatch mechanism, you can use the extra bit of separation between the store and store* to insert your logic)

emccue03:11:18

also this might be throwing a lot at you early on and definitely falls into the fiddly bits of designing a public library, not what you necessarily have to consider when making a prototype/first draft

emccue03:11:51

> This is being confusing since the bulk of documentation, including books, all want to show how in lein, which doesn’t deal with requiring from files the way files do. Also curious about this

emccue03:11:29

your question didn’t end up having to do with leiningen, but are you using that to build this or are you alluding to you using maven or deps.edn

emccue03:11:53

since if you are starting a project fresh nowadays the prevailing sentiment is to use deps.edn

emccue03:11:04

also super not important for a first draft sort of thing

Jack Park03:11:30

I'm starting a procedure and will return to this tomorrow morning. Many thanks for great information.

Jack Park03:11:40

Oh, I meant leiningen but I don't use it to test ideas the way docs do; I'm still in the early phases so I boot stuff I want to test by way of calls from core.cljs through 'lein run'

emccue03:11:45

ah so that is something to earmark

emccue03:11:09

the nicest way to use clojure isn’t really to run a file, make some changes, run a file, repeat

emccue03:11:05

generally speaking you load your project up and start a long running “REPL” process and write some code, load that code into the REPL, experiment with that code, repeat

emccue03:11:26

there is a bit of setup to that, but in the long run it is very worth it

emccue03:11:11

if you use emacs or vim there are dedicated wierdos out there for you, but there is also a very good IntelliJ plugin called Cursive and a very good VSCode plugin called Calva

Jack Park03:11:36

I'm working from intellij/cursive. I did what you say way back when I was hacking Forth some 30+ years ago; got away from that when I migrated through C to Java and got wrecked. I suspect that, in truth, I might be a bit lazy these days and do what moves more freely. I expect I'll get back to that method.

orestis08:11:19

Trying to see if something is doable in a macro. We have a (CLJS) API that takes string arguments, that are parts of a closed set. I would like to error out in compile time if an argument is passed in that is not part of that set. The problem obviously is that while I could define the closed set at compile time, the value that gets passed in is determined at runtime. Is there a clever trick I could use to accomplish this? E.g. an idea I had is to push the macro to the callsite where the arg is still a string literal, then it can be used. But it kinda breaks the ergonomics a bit.

p-himik08:11:28

If something isn't available at compile time, you can't use a macro for that. No clever tricks here. Usually such checks are done in run time. Or you can wrap the whole thing in a macro and require your users to pass strings directly into it, without using any intermediaries.

dataguy08:11:15

can the API provide introspection as data? (self-describing via an OPTIONS request is what I'm doing) then you could download (or, if it's a monorepo, just reference the API specs, e.g. malli definitions) during build time.

orestis09:11:11

@U2FRKM4TW thanks for confirming, a top-level macro was my hunch too.

orestis09:11:35

@UR5RAGFFG in our case it's just a static map of strings to strings, nothing fancier than that 🙂

1
emccue15:11:08

What you can do is use a macro to generate functions that will return the requests

emccue15:11:27

you will get a compile time check on whether the function you call exists

Ben Sless10:11:28

In reading the miniKanren thesis I found myself reading Kisleyov's papers on stream fusion, and suddenly, 🤯 those are transducers Moreover, this is what operators fusion does in RxJava, but with 1e6 lines instead of 6 lines like Clojure's transducers. Okay, cool. Then I thought back to my transducers workshop - can we extend reducibility to other things? Flowable can return iterable, then we can reduce it! We don't need all of RxJava, we already have transducers which handle operators fusion for us

👍 2
🌀 2
clojure-spin 2
Ben Sless10:11:31

(require '[clojure.core.protocols :as p])
(extend-protocol p/CollReduce
  Flowable
  (coll-reduce [coll f val]
    (p/coll-reduce (.blockingIterable coll) f val))
  Maybe
  (coll-reduce [coll f val]
    (p/coll-reduce (.blockingIterable (.toFlowable coll)) f val)))
That's it

emccue17:11:09

Is there a way i can get a consistent hash of uberjar builds? I want to skip a deploy step if nothing in a jar has changed

hiredman17:11:25

there are layers of trickiness there

hiredman17:11:46

one is that the clojure Compiler isn't always same bits in same bits out, so if you are including the output of that (aot compilation) in your jars that means you just can't hash things

hiredman17:11:39

uberjars are of course zip files, which contain things that change like metadata about files

hiredman17:11:05

the best thing to do is maybe hash sources before you build the uberjar

borkdude17:11:51

co-incidentally I'm working on a similar solution for CLJS advanced builds now

emccue17:11:09

maybe just hashing the contents of the files inside the jar

hiredman17:11:34

if you are aot compiling, that will include class files

emccue17:11:35

> the clojure Compiler isn’t always same bits in same bits out frustrating

hiredman17:11:08

hard to do when the compilation environment includes arbitrary code execution

Alex Miller (Clojure team)18:11:02

we do have one ticket related to this that I'm trying to move forward in 1.11 https://clojure.atlassian.net/browse/CLJ-1973

Alex Miller (Clojure team)18:11:33

probably not the only source of this, but I know it's one of the main sources of differences

Jacob Rosenzweig18:11:37

If I have a try catch and the last line of the catch is a (log/infof) function call, will that result to a nil in the return value?

Jacob Rosenzweig19:11:15

I believe the answer is "yes"

Jacob Rosenzweig18:11:01

Not sure if this will nil pun correctly.

hiredman19:11:39

I would not depend on the result of a logging call

☝️ 2
Jacob Rosenzweig19:11:33

So end the catch with a nil?

hiredman19:11:11

in general it will be nil, but tools.logging does some weird stuff sometimes

muhanga20:11:28

Can anyone point me in a way of "idiomatic" function return in case when I need to return one or two values. Should i look in the direction of "for"? Currently I fixed this by always returning collection that contain one or two elements, but I don't like it for some reason.

didibus20:11:14

Returning a vector or a map is quite idiomatic for those use cases

didibus20:11:44

But in general, you might think if there's a way to avoid the function returning one or two values? Maybe if you broke it into two functions?

didibus20:11:12

That said, sometimes it's what you have to do, in which case just a vector or map is pretty straightforward.

didibus20:11:43

But what I'd say is good practice, is if you find yourself doing something like: (if (= 1 (count return)) .. ..) Instead use a variant which would be something like: You'd return: [:color :green] or [:rgb 123 234 111] instead of :green or [123 234 111] Or with a map: {:variant :color-name, :color :green} or {:variant :rgb :red 123 :green 234 :blue 111} Basically a variant is a self-typed vector or map. The variant tells you what structure the vector or map has, so someone can branch on the variant to know which variant of vector or map it got. Its very similar to types, except the real type is still just vector or map. In fact, sometimes I call it :type instead of :variant.

hiredman20:11:55

Or taking an f argument and returning the result of calling f with one or two args

hiredman20:11:51

(pass in a continuation)

muhanga20:11:23

Interesting thoughts may try to return a map instead or a variant. This problem mostly happen when i try to parse a multiple thing from source in one go. It might be better to really split it into separate function. Thank you.

phronmophobic20:11:36

you can also return a map. eg. {:a 42} or sometimes {:a 42 :aa 43}