Fork me on GitHub
#clojure
<
2022-10-15
>
jjttjj12:10:51

Suddenly curious about this: let's say you have a random small one off task you want to do with clojure that requires dependencies and you sort of want to keep that code around for reference in the future, how do you do this? Make a whole new clojure project with deps.edn? Do you have a dir for a bunch of these types of things with a common deps.edn file? Do you have an alias in your root deps.edn that has a bunch of common dependencies? On a micro level, do you use ns or just (require 'x ...) and why? A recent example for me was there was a blog series with a title of contents page of all the posts, and I wanted to slurp that page and then slurp all the href links on it and concat the bodies so it's one big html file. How do you organize these little things? I'm trying to reconsider my dev workflow and organization and I'm curious what other people do

Ben Sless13:10:43

I have a junk project for every dependency I can think of for that. If something in it pans out I spin it off

Ben Sless13:10:04

But with neil it's so easy to create projects I started deviating from that

jpmonettas13:10:56

I have a clojure-playground deps.edn project with : deps.edn :

{:paths ["resources"]
 :deps {clj-commons/pomegranate                     {:mvn/version "1.2.1"}
        com.github.jpmonettas/flow-storm-inst       {:mvn/version "3.0.236"}
        com.github.jpmonettas/flow-storm-dbg        {:mvn/version "3.0.236"}}}
user.clj :
(ns user
  (:require [cemerick.pomegranate :as deps-pomegranate]
            [flow-storm.api :as fsa]))

(defn add-deps [& lein-style-dependency]
  (deps-pomegranate/add-dependencies
   :coordinates `[~@lein-style-dependency]
   :repositories {"central" ""
                  "clojars" ""}))

(defn start-debugger []
  (fsa/local-connect {:theme :dark}))

(defn stop-debugger []
  (fsa/stop))
and lets say I wan't to test the hiccup library, then I create playground/hiccup.clj with :
(user/add-deps '[hiccup "1.0.5"])

(ns playground.hiccup-test
  (:require [hiccup.core :refer [html]]))

(html [:div 1])

jpmonettas13:10:20

The idea of hot loading dependencies is to keep deps.edn deps small so it starts fast, and then every playground file is self contained

Alex Miller (Clojure team)13:10:11

Just one note on using RELEASE version - this fools the classpath caching so once you have that, unless you change the deps.edn or -Sforce, you will never get new versions of those deps

jpmonettas13:10:45

yeah, I had been bitten by that before

jpmonettas13:10:15

edited after Alex comments

jjttjj14:10:02

Nice, coincidentally I have actually experimented recently with Neil, and, separately, using the add-libs tools.deps branch, both seem really nice for this type of stuff

borkdude14:10:17

If you have babashka-compatible code you can also use this:

(require '[babashka.deps :as deps])
(deps/add-deps '{:deps {medley/medley {:mvn/version "1.4.0"}}})

(require '[medley.core :as medley])

(prn (medley/index-by :id [{:id 1} {:id 2}]))

Lone Ranger15:10:09

Eric Normand had a good article about running a single .clj file like a .sh by adding some sort of preamble to the top of the file but I can’t find it, will check my files when I get home

Lone Ranger15:10:30

This was in the dark ages before Babashka

valtteri18:10:22

I’ve started organizing different scratch projects as polylith components. I stole the idea from someone here but don’t remember who. :thinking_face: It’s been working quite well.

Joshua Suskalo20:10:01

https://gist.github.com/ericnormand/6bb4562c4bc578ef223182e3bb1e72c5

#!/usr/bin/env sh
#_(
  #_DEPS is same format as deps.edn. Multiline is okay.
  DEPS='{:deps {seesaw/seesaw {:mvn/version "1.5.0"}}}'

  #_You can put other options here
  OPTS='-J-Xms256m -J-Xmx256m -J-client'

  exec clojure $OPTS -Sdeps "$DEPS" -M "$0" "$@"
)

(use 'seesaw.core)

(native!)

(def f (frame :title "Test"))

(-> f pack! show!)

Joshua Suskalo20:10:40

then you chmod the file and run it from bash

Joshua Suskalo20:10:13

This works because this top segment happens to be both valid bash and valid clojure

Joshua Suskalo21:10:41

I see woodstock already linked this too

Lone Ranger23:10:01

If I recall @U04V70XH6 helped me adapt this a little while ago to use the -X parameter so that you can use individual functions from a file but I happen to have that computer in a storage unit

Lone Ranger23:10:22

And apparently I don’t know how to back things up to the internets

emccue19:10:31

Does anyone know any tools for recording the network activity coming out of an application? I know tools.deps is based on maven aether and I could read the code to know whats happening behind the scenes but i'm looking to brute force reverse engineer how stuff like VersionRangeRequest work

jpmonettas20:10:59

if you are on linux maybe you can do something like strace -f -e trace=network clj 2>&1 | grep sin_addr to start clojure and see what it is connecting to. And then use wireshark to look at the traffic

emccue20:10:35

im on mac so maybe thats close enough

jpmonettas20:10:42

no idea about macos, but there are probably some gui apps that constantly monitor network activity per app

jpmonettas20:10:03

maybe you can do it just with wireshark, you can capture and filter out all the noise before starting the app, when you run the program you should mostly be seeing what the new process is sending

phronmophobic20:10:13

although tbh, trying to reverse engineer via spying on the network calls rather than reading docs or source seems like the hard way to do it

Alex Miller (Clojure team)21:10:44

It’s all just http calls, so an http proxy should be sufficient

Alex Miller (Clojure team)21:10:27

I believe it’s requesting the version metadata file from each repository, then doing a local computation

Alex Miller (Clojure team)21:10:08

Note that not all repos publish the file so this is a possibly incorrect computation

chucklehead21:10:05

For HTTP/S traffic something like https://mitmproxy.org/ might be easier to deal with than wireguard

emccue21:10:28

I ended up getting most of what i was looking for with clj -.debug=all src/main.clj 2> out

emccue21:10:07

its doing a strange thing where it tries two urls for the metadata

emccue21:10:21

1. maven2/dev/whatever/maven-metadata.xml fails 2. dev/whatever/maven-metadata.xml succeeds 3. then maven2/dev/whatever/0.0.1/whatever-0.0.1.pom succeeds 4. and it gets the artifact it wants from maven2/dev/whatever/0.0.1/whatever-0.0.1.jar

emccue21:10:24

so...interesting

emccue21:10:55

in the output it doesnt say the domain, but the package im testing is only published to central - so one of those urls. clojars might work differently

Alex Miller (Clojure team)21:10:56

I think due to history there are a variety of places that metadata can be?

emccue21:10:18

it would probably be too much to ask from the universe to assume that history is documented, right?

emccue21:10:15

I know from reading coursier that ivy repos are a thing

Alex Miller (Clojure team)21:10:25

I don’t know where that’s documented, but would search for “maven repository layout”

emccue21:10:28

> Or read the code Have been - its wierdly brutal

emccue21:10:22

mitmproxy is working great but its not picking up the traffic from clj which is interesting. maybe it doesnt see it as http requests for some reason?