Fork me on GitHub
#clojure
<
2019-10-11
>
akiel08:10:24

Hi. Is someone using OSGi in a Clojure project?

mitchelkuijpers08:10:12

Hi Yes we do for Atlassian Plugins

akiel08:10:18

Thanks. Can you please point me to a project?

mitchelkuijpers08:10:08

I cannot because we have a closed source project 😞

mitchelkuijpers08:10:16

But I can help you get up and running

mitchelkuijpers08:10:27

Is there something in particular that is not working?

akiel08:10:48

Ok. How do you create the bundle manifests?

mitchelkuijpers08:10:28

We use maven for our Clojure project and then we point to a activator

akiel08:10:42

No I just try to find projects actually using OSGi with Clojure. Because I can’t find current ones.

mitchelkuijpers08:10:57

like this:

<Bundle-Activator>nl.avisi.atlascrm.atlascompat.ClojureActivator</Bundle-Activator>

akiel08:10:10

I like to have a plugin system for a application I develop and evaluate to use OSGi for that.

mitchelkuijpers08:10:36

Is it a Clojure project for which you want to create a Plugin System?

akiel08:10:57

all Clojure app and plugins

mitchelkuijpers08:10:58

Then I would not go that route if I where you

akiel08:10:17

ok that is interesting - why?

mitchelkuijpers08:10:33

OSGI is pretty complex, and Clojure by default is not compatible

mitchelkuijpers08:10:57

So probably every Plugin ends up with their own Clojure runtime, which gives quite a lot of overhead

mitchelkuijpers08:10:10

Because every OSGI bunde bundles their own dependencies

mitchelkuijpers08:10:35

You could export some Clojure API to require things, but then you get problems with which Classloader should be used to load stuff

mitchelkuijpers08:10:06

It is just a rabbithole that I would not recommend, we only do this because we had a existing Clojure project and wanted to use that in our Atlassian Plugin

akiel08:10:08

ok that is a good point. because of possible dependency issues, I started to look at OSGi.

mitchelkuijpers08:10:32

Yeah that is a very logical thing to do

mitchelkuijpers08:10:47

There are some clojure-osgi projects which can load namespaces from other bundles

mitchelkuijpers08:10:07

But they all have weird problems with: AOT, clojure.tools.logging does not work

akiel08:10:16

If you just use uberjars for plugins and load them into your classpath, there will be dependency issues if you have different versions of the same dependency.

mitchelkuijpers08:10:17

And clojure.pprint fails on lot's of them

mitchelkuijpers08:10:41

If you want everyone to bundle their own Clojure version with all deps then it could work

akiel08:10:32

I really havent thought about needing a separate Clojure runtime for every plugin. So you are right, that would be just to much overhead.

mitchelkuijpers08:10:29

No I have never looked at that tbh

mitchelkuijpers08:10:26

The metabase thing looks pretty interesting though

akiel08:10:01

Yes I will try to use conflicting dependencies with the metabase classloading system.

mitchelkuijpers08:10:31

Sorry I don't have any better news about the OSGI front 😞

mitchelkuijpers08:10:40

But hopefully this will save you a lot of time

akiel08:10:51

So thank you. You have saved me some time by not looking deeply into OSGi.

mitchelkuijpers08:10:45

If you get anywhere with the metabase approach I would find that very interesting

jthomson13:10:16

FWIW we got OSGi working with Clojure as a shared dependency using https://github.com/talios/clojure.osgi - which includes an example of creating a manifest with maven-bundle-plugin

jthomson13:10:07

the only class-loading issue we ran into was stale namespace state, which was solved with :reload-all

borkdude17:10:11

what is a good way to pass a callable thing to Clojure from Java? E.g. a function of two arguments. Passing lambda's doesn't work. Implementing an IFn is awkward since none of the methods are marked as default.

leonoel17:10:59

subclass AFn, it has concrete default implementations

borkdude17:10:15

@hiredman those things don't implement IFn

hiredman17:10:32

but just all the method instead of invoking

borkdude18:10:44

Thanks. AFn works for me

Ilya19:10:11

Hello, I have some data in a vector and for each element, I want to see if it has something in common with the rest of the elements after it. What would you think of reduce for this? Since I’m looking at the rest of the items, not just the next one, it seems a little unnatural…

Ilya19:10:27

I’m sorry if this is too generic

jaihindhreddy07:10:23

A couple of example inputs and expected outputs would make it very clear 🙂

jaihindhreddy07:10:23

Usually finding a specific problem like that, will give you specific solutions which contain the general ideas

Ilya19:10:11

Maybe I can make do with loop but was wondering if there’s something more “higher level”

vlaaad19:10:32

@ilyab partition-by:

(partition-by :a [{:a 1} {:a 1} {:a 2} {:a 3}])
=> (({:a 1}
     {:a 1})
    ({:a 2})
    ({:a 3}))

andy.fingerhut19:10:29

I'm taking a stab in the dark, but if you reversed the vector, and then did reduce on that, the 'accumulator' data that you pass from one step to the next could be something about all of the reversed elements seen so far, and compared against the next one?

noisesmith19:10:55

another common operation for comparing to other inputs is group-by

noisesmith19:10:43

user=> (group-by :a [{:a 1} {:a 1} {:a 2} {:a 3}])
{1 [{:a 1} {:a 1}], 2 [{:a 2}], 3 [{:a 3}]}

dpsutton19:10:41

set/index might offer some idea as well?

dpsutton19:10:49

although there's no notion of "after"

Ilya19:10:07

Thanks, this gives me some ideas!

peterdee21:10:47

Some strange behavior perhaps someone can explain. I have a large-ish data structure (vector containing maps) that raises an exception in my program when I use it, but doesn’t cause an exception when I first stringify it and do read-string on it. The exception itself doesn make any sense either, it is

Execution error (ExceptionInfo) at datahike.db/validate-val (db.cljc:989).
Bad entity value 0 at [:db/add 58 :ImportFrom/level 0], value does not match schema definition. Must be conform to: (= (class %) java.lang.Long)
I think 0 is a long.

peterdee21:10:04

My code

(defn tryme []
  (let [ex (code2ast (slurp "resources/example.py"))
        schema  (->> ex
                     (learn-schema (schema-as-map base-schema))
                     schema-as-schema)]
    (d/delete-database uri) ; cleanup previous database
    (d/create-database uri :initial-tx schema) ; create the in-memory database
    (let [conn (d/connect uri)] ; connect to it (returns an atom)
      (d/transact conn ex #_(read-string (cl-format nil "~S" ex)))
      (simple-example conn))))
You can see where I’ve commented out the read-string to use ex in this example.

ghadi21:10:08

@peterd examine the exception, which appears to be a data-bearing exception, by calling ex-data on it

ghadi21:10:42

In it, you may see something odd about the problematic value

ghadi21:10:01

You can usually grab the exception itself with`*e`

peterdee21:10:08

> (try (tryme) (catch Exception e (ex-data e)))
{:error :transact/schema,
 :value 0,
 :attribute :ImportFrom/level,
 :schema
 #:db{:valueType :db.type/long,
      :ident :ImportFrom/level,
      :cardinality :db.cardinality/one}}
I tried doing it like the above.

peterdee21:10:50

Here it is in detail

#error {
 :cause "Bad entity value 0 at [:db/add 58 :ImportFrom/level 0], value does not match schema definition. Must be conform to: (= (class %) java.lang.Long)"
 :data {:error :transact/schema, :value 0, :attribute :ImportFrom/level, :schema #:db{:valueType :db.type/long, :ident :ImportFrom/level, :cardinality :db.cardinality/one}}
 :via
 [{:type clojure.lang.ExceptionInfo
   :message "Bad entity value 0 at [:db/add 58 :ImportFrom/level 0], value does not match schema definition. Must be conform to: (= (class %) java.lang.Long)"
   :data {:error :transact/schema, :value 0, :attribute :ImportFrom/level, :schema #:db{:valueType :db.type/long, :ident :ImportFrom/level, :cardinality :db.cardinality/one}}
   :at [datahike.db$validate_val invokeStatic "db.cljc" 989]}]
 :trace
 [[datahike.db$validate_val invokeStatic "db.cljc" 989]
  [datahike.db$validate_val invoke "db.cljc" 981]
  [datahike.db$transact_add invokeStatic "db.cljc" 1200]
  [datahike.db$transact_add invoke "db.cljc" 1198]
  [datahike.db$transact_tx_data invokeStatic "db.cljc" 1434]
  [datahike.db$transact_tx_data invoke "db.cljc" 1280]
  [datahike.core$with invokeStatic "core.cljc" 231]
  [datahike.core$with invoke "core.cljc" 224]
  [datahike.core$_transact_BANG_$fn__41228 invoke "core.cljc" 438]
  [clojure.lang.Atom swap "Atom.java" 37]
  [clojure.core$swap_BANG_ invokeStatic "core.clj" 2352]
  [clojure.core$swap_BANG_ invoke "core.clj" 2345]
  [datahike.core$_transact_BANG_ invokeStatic "core.cljc" 437]
  [datahike.core$_transact_BANG_ invoke "core.cljc" 434]
  [datahike.core$transact_BANG_ invokeStatic "core.cljc" 529]
  [datahike.core$transact_BANG_ invoke "core.cljc" 444]
  [datahike.core$transact invokeStatic "core.cljc" 636]
  [datahike.core$transact invoke "core.cljc" 629]
  [datahike.core$transact invokeStatic "core.cljc" 633]
  [datahike.core$transact invoke "core.cljc" 629]
  [datahike.connector$eval55768$fn__55770$fn__55772 invoke "connector.cljc" 31]
  [clojure.core$binding_conveyor_fn$fn__5754 invoke "core.clj" 2030]
  [clojure.lang.AFn call "AFn.java" 18]
  [java.util.concurrent.FutureTask run "FutureTask.java" 264]
  [java.util.concurrent.ThreadPoolExecutor runWorker "ThreadPoolExecutor.java" 1128]
  [java.util.concurrent.ThreadPoolExecutor$Worker run "ThreadPoolExecutor.java" 628]
  [java.lang.Thread run "Thread.java" 834]]}
I don’t see a problem with that.

peterdee21:10:52

I can print the variable to the cider REPL, assign it to a var, and likewise it runs fine, just like doing (read-string (cl-format nil "~S~" var)

andy.fingerhut21:10:57

If you can store that map in a Clojure Var at a REPL, or in your program, and print out (class <expression-that-extracts-the-number-0>), it would be interesting to know whether it was java.lang.Long or some other type.

andy.fingerhut21:10:20

It could be java.lang.Integer, for example, and would look exactly the same in that output either way.

peterdee21:10:52

I’ll try that now.

andy.fingerhut21:10:07

Sorry, I should have said vector instead of map, but you get the idea.

andy.fingerhut21:10:53

Printing out a java.lang.Integer type value (or Byte, or Short, etc.) and reading it back in would definitely change it to a Long

peterdee21:10:26

I am having a problem with expression-that-extracts-the-number-0. #error {…} I though (-> foo :error/data ….)

peterdee21:10:44

though -> thought

andy.fingerhut21:10:22

Are you able to capture it in a REPL? That lets you retry multiple variations more quickly, perhaps.

andy.fingerhut21:10:16

And the key might be just :data, not :error/data?

peterdee21:10:29

Yeah, (type foo) clojure.lang.ExceptionInfo

noisesmith21:10:10

you need the ex-data function to get the data map from the Exception object

peterdee21:10:13

(-> foo ex-data :value class)
--> java.lang.Integer

peterdee21:10:28

I’m learning a lot here. Thanks!

andy.fingerhut21:10:45

Two possibilities suggest themselves -- figure out where the Integer type value is coming from, and find a convenient place to convert it to Long. Or, find out who wrote the spec that checks to see if it is Long, and see if they can loosen it to allow other integer types.

peterdee21:10:46

Thank you both.

andy.fingerhut21:10:02

There is a #datahike channel where people familiar with that spec might hang out.

peterdee21:10:42

Yes. I’ll do that, thanks!

Alex Miller (Clojure team)21:10:55

int? is a pred for fixed size integers. integer? is a pred for any integer (fixed or arbitrary precision)

Alex Miller (Clojure team)21:10:05

in case those are helpful

peterdee21:10:25

I’ll let the spec writer know!

peterdee21:10:51

Or send a PR

andy.fingerhut21:10:38

They might have efficiency concerns about allowing other integer types and doing the conversion on the library side, but I suspect you will find out when interacting with the datahike devs.

peterdee22:10:30

They allow these:

#{:db.type/instant :db.type/boolean :db.type/uuid
   :db.type/value :db.type/string :db.type/keyword :db.type/ref
   :db.type/bigdec :db.type/float :db.type/bigint :db.type/double
   :db.type/long :db.type/symbol}

peterdee22:10:56

So I guess loosening up :db.type/long, just calling it :db.type/int might be best.

Dale22:10:28

I've got a function that needs to return a map with, say, twenty calculations, and most of the values in the map are dependent on one of the other values. Right now I've got something like this:

(defn f [...]
  (let [foo ...
        bar (something-with foo)
        baz (something-else bar)
        ...]
    {:foo foo
     :bar bar
     :baz baz
     ...}))
Is there a better way to write this? As you can see, I'm currently repeating foo, bar, etc. three times where I'd prefer it to be repeated just once.

potetm15:10:16

IMO - this is better than special threaders. The dependencies are clear. The desired execution order is clear. The way the code is structured tells the reader useful information.

potetm15:10:29

I see no value whatsoever in golfing this down.

seancorfield22:10:11

@dale I might reach for as-> in this case and thread the "map built so far" into each calculation that can add each new computed var

Dale22:10:13

I'm just finding suggestions to write a macro using &env, but I'm wondering if there's a better way without resorting to macros.

seancorfield22:10:41

(normally I advocate as-> inside -> but here I think it's worth making an exception)

peterdee22:10:15

@seancorfield beat me to it. I would use as->

seancorfield22:10:36

(as-> {} m
      (assoc m :foo (compute-foo))
      (assoc m :bar (compute-bar (:foo m)))
      ,,,)

Dale22:10:30

@seancorfield, @peterd That is just what I was looking for, thanks!

seancorfield22:10:08

I might still start it with -> TBH

(-> {}
    (assoc :foo (compute-foo))
    (as-> m
          (assoc m :bar (compute-bar (:foo m)))
          ,,,))

peterdee23:10:56

I picked up the habit of using a ? in the variable. Thus (as-> {:foo (compute foo)} ?m (assoc ?m,,,))

seancorfield23:10:55

I'm torn on that. I tend to use short, mnemonic names (`m` for map, for example), but I do have sympathy for just using symbols, such as % or $...

ahungry23:10:55

What benefit is there to start with the plain arrow?

peterdee23:10:32

In the example, I’m not sure. You’d like to bind a name and carry it through each form, thus (as->...)