Fork me on GitHub
#datomic
<
2020-02-02
>
mdhaney17:02:13

This file lists all the namespaces to suppress for Datomic. This configuration is for timbre - if you are using a different logging framework, the configuration will be different. https://github.com/fulcrologic/fulcro-template/blob/master/src/main/config/defaults.edn

Mehdi H.09:02:24

Hey, thanks mdhaney! I see none of these options in the logback file, so I can't really mute them. For instance I'd definitely mute the datomic.process-monitor one, but I can't see it. I tried and commented everything in it : the process-moniitor logs are still visible in my REPL...

Mehdi H.10:02:10

So if anyone knows how to make the logs from datomic.process-monitor disappear from my REPL (it gets in the way of my typing in it a lot and it's really annoying...) with a basic datomic-pro on-prem logback configuration, I'd be more than happy to hear from you! 🙂

Mehdi H.20:02:18

Thanks @U05120CBV! Usually I do clj -A:rebel:dev in a terminal with :

:dev {:extra-paths ["dev"]
                 :extra-deps {
                              org.clojure/tools.namespace {:mvn/version "0.3.1"}
                              org.clojure/java.classpath {:mvn/version "0.3.0"}
                              }}
           :rebel {:extra-deps {com.bhauman/rebel-readline {:mvn/version "0.1.4"}}
                    :main-opts  ["-m" "rebel-readline.main"]}

Mehdi H.20:02:45

But I seem to recall the logs were also "polluting" the REPL with the Emacs CIDER integration for instance.

marshall20:02:36

Whats in your logback.xml?

Mehdi H.21:02:53

I have commented most of the lines and the logs from datomic.process-monitor are still coming to the REPL :

<configuration>

  <!-- prevent per-message overhead for jul logging calls, e.g. Hornet -->
  <contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
    <resetJUL>true</resetJUL>
  </contextListener>

  <appender name="MAIN" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>${DATOMIC_LOG_DIR:-log}/%d{yyyy-MM-dd}.log</fileNamePattern>
      <maxHistory>72</maxHistory> 
    </rollingPolicy>
    <prudent>true</prudent> <!-- multi jvm safe, slower -->
    <encoder>
      <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level %-10contextName %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>

 <!--  <logger name="datomic.cast2slf4j" level="DEBUG"/> -->

  <!-- uncomment to log storage access -->
  <!-- <logger name="datomic.kv-cluster" level="DEBUG"/> -->

  <!-- uncomment to log transactor heartbeat -->
  <!-- <logger name="datomic.lifecycle" level="DEBUG"/> -->

  <!-- uncomment to log transactions (transactor side) -->
  <!-- <logger name="datomic.transaction" level="DEBUG"/> -->

  <!-- uncomment to log transactions (peer side) -->
  <!-- <logger name="datomic.peer" level="DEBUG"/> -->

  <!-- uncomment to log the transactor log -->
  <!-- <logger name="datomic.log" level="DEBUG"/> -->

  <!-- uncomment to log peer connection to transactor -->
  <!-- <logger name="datomic.connector" level="DEBUG"/> -->

  <!-- uncomment to log storage gc -->
  <!-- <logger name="datomic.garbage" level="DEBUG"/> -->

  <!-- uncomment to log indexing jobs -->
  <!-- <logger name="datomic.index" level="DEBUG"/> -->

  <!-- these namespsaces create a ton of log noise -->
   <!-- <logger name="httpclient" level="INFO"/>
  <logger name="org.apache.commons.httpclient" level="INFO"/>
  <logger name="org.apache.http" level="INFO"/>
  <logger name="org.jets3t" level="INFO"/>
  <logger name="com.amazonaws" level="INFO"/>
  <logger name="com.amazonaws.request" level="WARN"/>
  <logger name="sun.rmi" level="INFO"/> 
  <logger name="net.spy.memcached" level="INFO"/>
  <logger name="com.couchbase.client" level="INFO"/>
  <logger name="org.apache.zookeeper" level="INFO"/>
  <logger name="com.ning.http.client.providers.netty" level="INFO"/>
  <logger name="org.eclipse.jetty" level="INFO"/>
  <logger name="org.hornetq.core.client.impl" level="INFO"/>
  <logger name="org.apache.tomcat.jdbc.pool" level="INFO"/> -->

 <!--  <logger name="datomic.cast2slf4j" level="DEBUG"/> -->

  <root level="info">
    <appender-ref ref="MAIN"/>
  </root>
</configuration>

Mehdi H.21:02:09

This above is the logback from bin/ in the datomic folder. There is no other logback.xml file in the folder of the project.

Mehdi H.13:02:57

After checking, even if I just launch `

clj 
in the terminal, and request a connection to a db uri, the logs from datomic.process-monitoring keep on coming. Any thoughts @U05120CBV?

marshall15:02:54

try commenting on the appender entirely this line <appender-ref ref=“MAIN”/>

marshall15:02:23

you may have a global system redirect somewhere on your local machine that’s resulting in terminal appender instead of file

Mehdi H.15:02:08

Ok makes sense! Trying this immediately!

Mehdi H.15:02:34

hmm so I just commented the one line you mentioned, leaving <root level="info" etc, but no, still coming relentlessly to my REPL.

Mehdi H.15:02:09

I am on WSL (ubuntu 18.04) on an otherwise Windows 10 laptop. I just don't see where I would have created the redirect that you mention. Normally I get from the rest of the file that I should get these logs as files with a 3 day rolling history. I am going to look for those files in the install folder.

Mehdi H.15:02:20

yep I do get daily log files in the log folder as well... Am I looking at a conflict of logging policies, would you say that could cause the double logging?

marshall15:02:30

try taking logback out of your peer app dependencies

marshall15:02:45

what all is in your deps.edn?

marshall15:02:04

if you have no logback in your classpath, it will default to logging in your terminal

marshall15:02:10

this is separate from the transactor logs

marshall15:02:22

which is what the logback in the datomic distribution controls

marshall15:02:38

i would put the configs back they way they were in the datomic logback

marshall15:02:56

then create a logback in your peer application (or remove the logback dependency from your deps.edn)

Mehdi H.15:02:23

This is my deps.edn file of the project:

{:mvn/repos
 {"" {:url ""}}
 :deps
 {org.clojure/clojure {:mvn/version "1.10.1"}
  buddy/buddy-auth {:mvn/version "2.2.0" :exclusions [cheshire]}
  com.datomic/datomic-pro {:mvn/version "0.9.5951" :exclusions [org.slf4j/slf4j-nop]}
  com.stuartsierra/component {:mvn/version "0.4.0"}
  org.slf4j/slf4j-simple {:mvn/version "1.7.28"}
  org.clojure/tools.logging {:mvn/version "0.5.0"}
  com.stuartsierra/component.repl {:mvn/version "0.2.0"}
  io.pedestal/pedestal.service {:mvn/version "0.5.7"}
  io.pedestal/pedestal.route   {:mvn/version "0.5.7"}
  io.pedestal/pedestal.jetty   {:mvn/version "0.5.7"}
  com.draines/postal {:mvn/version "2.0.3"}
  clj-template {:mvn/version "1.0.1"}
  com.velisco/clj-ftp {:mvn/version "0.3.12"}
  buddy/buddy-sign {:mvn/version "3.1.0"}
  clj-time {:mvn/version "0.15.2"}
  tupelo-datomic {:mvn/version "0.9.2"}}
 :paths ["src" "resources"]}

Mehdi H.15:02:58

So basically I was messing with the wrong file, ok, I'll put it back to what it was before

Mehdi H.15:02:01

So you are saying the datomic.process-monitoring logs are different from the transactor logs. Does that mean that whatever dependency I have for a logging framework in my classpath is doing all this?

Mehdi H.15:02:34

I see this sentence on the github page of tools.logging : "You can redirect all java writes of `System.out` and `System.err` to the log system by calling `log-capture!`

Mehdi H.15:02:22

Is it me misusing tools.logging or slf4j as I see a lot of slf4j dependencies in a clj -Stree command...?

marshall15:02:45

you have org.slf4j/slf4j-simple {:mvn/version “1.7.28”} in your des

marshall15:02:06

if you have slf4j in your deps and no logback.xml in your classpath you will get default logging behavior from datomic peer library

marshall15:02:09

to your console

marshall15:02:23

you can either remove that dep or make a logback file with the configs you actually want

Mehdi H.15:02:41

Oh I see! I wanted to use slf4j because I think I felt I had to choose something like that to use tools.logging.

Mehdi H.15:02:32

So I need to build a logback file and put it on my classpath. Makes perfect sense! Many thanks for taking the time to debug this rookie issue @U05120CBV! Much appreciated!