KiCad with generated net and cmp files

From Electriki
Revision as of 12:48, 6 December 2015 by Michai (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search


This text describes how to generate KiCad netlists and component-assignments files programmatically using a domain-specific language (DSL) on top of Tcl.

Background

In an earlier page, ways in which KiCad eeschema netlist- (.net) and cvpcb component-files (.cmp) can be simplified were given.

The idea then was to eventually be able to generate both these files programmatically, thus skipping the schematic capture and component-assignment stages.

This can be useful when a project contains repetitive or hierarchical circuitry. Although KiCad itself has a concept of "hierarchical circuit", it can be cumbersome having to spend space on schematic-pages for very similar circuitry, and having to provide glue-wiring between these blocks.

Furthermore, I don't think changes in hierarchical circuits propagate up to already existing instances - I don't think hierarchical circuits were meant to be used like macros, but more as a way to abstract a single subcircuit at a time.

Instead, circuits and subcircuits can be modeled using something resembling procedures and macros in programming languages - which is what this page is about.

Caveat

  • It's not possible to draw part of a circuit using eeschema, and generating another part of that circuit programmatically.
  • Semantic checks (DRC) provided by eeschema are unavailable. Furthermore, at this point the generator is very basic, lacking most error-checking. It's really easy to shoot yourself in the foot.
  • It's probably easier to quickly grasp the workings of a circuit visually, than it is to read it in text-/code-form. The latter method shines when there's a lot of relatively simple circuitry.
  • The current generation-code is not very clean. I'm still learning to use Tcl, and this tool acts as a testbed for myself. Focus should be on the DSL and new workflow instead :-)
  • This is a proof-of-concept. Circuit-definitions are mixed with instructions to generate the actual netlist and component-assignments file. Eventually, the latter may be moved to a driver.

Merged schematic capture and component-assignment

In the KiCad workflow, eeschema is the schematic capture tool, and cvpcb is the component-assignment tool.

Schematic symbols are in principle separate from corresponding physical packages. Associating footprints with schematics symbols works by means of matching pin-numbers.

In generating everything up to the layout-phase programmatically, I chose to merge schematic symbol and assigned footprint in something called a "physical component".

The idea here is that abstraction can be used instead, to accomplish a better separation between footprints and building-blocks used for modeling. (See the NPN-transistor abstraction later on for an example.)

Learn by example: an inverter out of discrete components

A logic inverter can be constructed out of resistors and a transistor as follows:

caption inverter out of discrete components

In the normal KiCad workflow, the resistor and transistor could be drawn in a hierarchical circuit, which could then be instantiated multiple times.

Using the tool described here, it can be described programmatically instead:

    virtual component "inverter" with pins { A Y Vcc GND } consists of {
    
        resistor_1k "Rc" { 
            { pin 1 at Vcc } 
            { pin 2 at Y   } }
    
        resistor_100 "Rs" { { pin 1 at A } }
    
        transistor_NPN "Q" { 
            { pin b at Rs:2 }
            { pin c at Y    }
            { pin e at GND  } }
    }

This defines a new part called "inverter" to be used in a circuit-description. (I chose for a verbose sort of language, with risk of making it look like LOLCODE.)

Explanation:

  • the given pin-list (A, Y, Vcc, GND) is currently not used, but is intended for clarity and future error-checking. The pin-list is in a way analogous to a set of formal parameters for a procedure in common programming languages.
  • an inverter consists of subcomponents: 2 resistor-instances (Rs and Rc) and a transistor-instance (Q) - see the drawing given earlier
  • each subcomponent is connected with each of its local pins either to
    • a virtual pin of the compound component (e.g. Rc's pin 1 is connected to Vcc in the inverter's pin-list, using statement "{ pin 1 at Vcc }"), or
    • a pin of a previously instantiated subcomponent (e.g. Q's base (b) is connected to pin 2 of previously instantiated Rs, using statement "{ pin b at Rs:2 }").

So, the "inverter" clause creates a new super-component as shown in the picture.

Physical and virtual components

The inverter defined above is called a virtual component - simply said, it does not occur in the BOM, because it consists of other components.

Although virtual components can consist of other virtual components (e.g. an oscillator containing among other parts an inverter), at the lowest level, only so-called physical components make up the circuit.

In a circuit-description, a physical component has no statements describing its inner workings, simply because it's already "finished" as-is. A physical component has a value (currently not used, but meant for clarity and BOM-generation later) and a footprint (e.g. "SM0603" in KiCad-lingo).

As an example, in the description of the inverter above, a physical component "resistor_100" is instantiated as base-resistor (Rs):

    ...                    
    resistor_100 "Rs" { { pin 1 at A } }
    ...                    

The definition of "resistor_100" is as follows:

    physical component "resistor_100" with pins { 1 2 } has value 100 and footprint "SM0603"

This should speak for itself: "resistor_100" is a 100 Ohm 0603 resistor, to be instantiated elsewhere.

Encapsulation to hide footprint-details

The inverter-example uses a NPN-transistor (Q) as switching element.

In KiCad

This is where KiCad starts to leak: during schematic capture, the user doesn't want to be concerned with the physical choice of transistor, and likely wants to use a generic "NPN transistor" symbol for all NPN-transistors. This symbol has its pins numbered from 1 to 3.

However, different physical transistors have different packages, and even within a package (e.g. SOT-23), transistor-terminals may not always be exposed at the same package-lead.

Since there is no additional layer mapping schematic symbols to footprints, this means that either multiple schematic symbols or multiple footprints have to be used, with only difference between them being the pin-assignment. (Having multiple footprints probably makes more sense than having multiple schematic symbols here.)

DISCLAIMER: I may be misunderstanding the KiCad-way, but fact is that there exist e.g. different "SOT-23" footprints to accomodate for differences in pin-numbering. I can't see any way of making this work without an additional mapping-layer, which KiCad to my knowledge does not have.

In this tool...

...the choice was made to create that missing abstraction-layer. That is, physical components can be wrapped inside another component-description with the only effect being a renumbering or renaming of the pins.

As an example, the given inverter was described using a generic "transistor_NPN" component:

    ...
    transistor_NPN "Q" { 
        { pin b at Rs:2 }
        { pin c at Y    }
        { pin e at GND  } }
    ...

This subcomponent has pins named b, c and e with obvious meanings, and is described as follows:

    virtual component "transistor_NPN" with pins { b c e } consists of {
    
        bc847 "Q" { 
            { pin 1 at b }
            { pin 2 at e }
            { pin 3 at c } }
    }

Thus, underlying each instance of "transistor_NPN" is a physical component (BC847):

    physical component "bc847" with pins { 1 2 3 } has value "bc847" and footprint "SOT23"

The pin-numbering of this physical component follows the actual pin-numbering as given in its data-sheet, as can be seen here:

caption transistor abstraction

So virtual component "transistor_NPN" hides the actual pin-numbering of the physical component. In case another physical transistor is used instead, only the definition of "transistor_NPN" has to be changed accordingly.

(This method has shortcomings too, but works reasonably well for simple but large schematics.)

Complete description of inverter

The complete description of inverter, followed by a single instantiation (as U1) and instructions to write netlist and component-assignments file looks like this:

    physical component "resistor_100" with pins { 1 2 } has value 100 and footprint "SM0603"
    
    physical component "resistor_1k" with pins { 1 2 } has value "1k" and footprint "SM0603"
    
    physical component "bc847" with pins { 1 2 3 } has value "bc847" and footprint "SOT23"
    
    
    
    virtual component "transistor_NPN" with pins { b c e } consists of {
    
        bc847 "Q" { 
            { pin 1 at b }
            { pin 2 at e }
            { pin 3 at c } }
    }
    
    
    
    virtual component "inverter" with pins { A Y Vcc GND } consists of {
    
        resistor_1k "Rc" { 
            { pin 1 at Vcc } 
            { pin 2 at Y   } }
    
        resistor_100 "Rs" { { pin 1 at A } }
    
        transistor_NPN "Q" { 
            { pin b at Rs:2 }
            { pin c at Y    }
            { pin e at GND  } }
    }
    
    
    
    physical component "testpad" with pin 1 has value "test" and footprint "TESTPAD"
    
    
    
    # ##########################################################################################
    
    
    
    testpad "P1" { { pin 1 at power } }
    testpad "P2" { { pin 1 at input } }
    testpad "P3" { { pin 1 at output } }
    testpad "P4" { { pin 1 at ground } }
    
    inverter "U1" { 
        { pin A   at input  }
        { pin Y   at output }
        { pin Vcc at power  }
        { pin GND at ground } }
    
    
    
    write_kicad_netlist /tmp/kicad_test/kicad_test.net
    write_kicad_cmplist /tmp/kicad_test/kicad_test.cmp

What's great about this, is that this is valid Tcl code! In other words, parsing of the DSL is done implicitly.

Generated netlist and component-assignments file

The resulting netlist (_*.net_) generated by the tool looks as follows:

    (export (version D)
    (components
    (comp (ref P1) (value test))
    (comp (ref P2) (value test))
    (comp (ref P3) (value test))
    (comp (ref P4) (value test))
    (comp (ref U1_Rc) (value 1k))
    (comp (ref U1_Rs) (value 100))
    (comp (ref U1_Q_Q) (value bc847))
    )
    (nets
    (net (code 1) (name "power")
    (node (ref P1) (pin 1))
    (node (ref U1_Rc) (pin 1))
    )
    (net (code 2) (name "input")
    (node (ref P2) (pin 1))
    (node (ref U1_Rs) (pin 1))
    )
    (net (code 3) (name "output")
    (node (ref P3) (pin 1))
    (node (ref U1_Rc) (pin 2))
    (node (ref U1_Q_Q) (pin 3))
    )
    (net (code 4) (name "ground")
    (node (ref P4) (pin 1))
    (node (ref U1_Q_Q) (pin 2))
    )
    (net (code 5) (name "")
    (node (ref U1_Rs) (pin 2))
    (node (ref U1_Q_Q) (pin 1))
    )
    )
    )

As can be seen, top-level names from the circuit-description such as "output" and "ground" occur as net-names in the netlist. This helps during the layout-stage. (Internal nets are not named, but each net is always given a unique code.)

The refdes-naming (e.g. "U1_Rs") gives a hint as to the location of the component in the circuit-hierarchy. This also helps while layouting, making it easier to group nearby components together.

Resulting component-assignments file is as follows:

    Cmp-Mod V01
    
    BeginCmp
    Reference = P1;
    IdModule  = TESTPAD;
    EndCmp
    
    BeginCmp
    Reference = P2;
    IdModule  = TESTPAD;
    EndCmp
    
    BeginCmp
    Reference = P3;
    IdModule  = TESTPAD;
    EndCmp
    
    BeginCmp
    Reference = P4;
    IdModule  = TESTPAD;
    EndCmp
    
    BeginCmp
    Reference = U1_Rc;
    IdModule  = SM0603;
    EndCmp
    
    BeginCmp
    Reference = U1_Rs;
    IdModule  = SM0603;
    EndCmp
    
    BeginCmp
    Reference = U1_Q_Q;
    IdModule  = SOT23;
    EndCmp
    
    EndListe

LOLworthy anecdote: apparently my KiCad-version (2013-may-18 stable) cares about whitespace.

This worked...

    IdModule  = SM0603;

...while this did not:

    IdModule = SM0603;

(spot the difference). I am guessing this is fixed in more recent versions.

Resulting ratsnest

The ratsnest when reading these files into pcbnew looks like this after moving components around a bit:

caption layout from generated files

Generator source-code

Not too pretty (lots of globals and very imperative), but it works:

    #!/usr/bin/env tclsh
    
    
    
    variable nets {}
    
    
    
    proc make_proc { name arglist body args } {
    
        set body [ string map $args $body ]
    
        proc $name $arglist $body
    }
    
    
    
    proc val_or_default { _var default } { 
    
        expr { [ uplevel #0 info exists $_var ] ? [ uplevel #0 set $_var ] : $default } 
    }
    
    
    
    proc lpop _li {
    
        upvar 1 $_li li
    
        set li [ lreplace $li end end ]
    }
    
    
    
    proc refdes_path_get {} { val_or_default refdes_path {} }
    
    
    
    proc refdes_path_push refdes { uplevel #0 lappend refdes_path $refdes }
    
    
    
    proc refdes_path_pop {} { uplevel #0 lpop refdes_path }
    
    
    
    proc component_add_virtual refdes {
    
        global components
    
        dict set components $refdes {}
    }
    
    
    
    proc component_add_physical { refdes value footprint } { 
    
        global components
    
        dict set components $refdes value     $value
        dict set components $refdes footprint $footprint
    }
    
    
    
    proc set_component_footprint { compname footprint } { uplevel #0 dict set component_footprints $compname $footprint }
    
    
    
    proc add_connection_to_netlist { new_pin existing_pin } {
    
        global nets
    
        for { set i 0 } { $i < [ llength $nets ] } { incr i } {
    
            if { $existing_pin in [ lindex $nets $i ] } {
    
                # Net for existing pin found; add new pin to that net.
    
                lset nets $i end+1 $new_pin
    
                return
            }
        }
    
        # No net for existing pin exists yet; create net, and add both existing and new pins to it.
    
        lappend nets [ list $new_pin $existing_pin ]
    }
    
    
    
    proc make_connections { connections refdes } {
    
        set parent_refdes_path [ refdes_path_get ]
        refdes_path_push $refdes
    
        foreach connection $connections {
    
            # Local pin always exists in the component being instantiated.
            
            set local_pin [ concat [ refdes_path_get ] [ lindex $connection end-2 ] ]
    
            # Existing pin can exist in previously defined sibling or in parent.
    
            set pin_string [ lindex $connection end ]
    
            if { [ regexp {^(\w+):(\w+)$} $pin_string -> sibling sib_pin ] } {
    
                set existing_pin [ concat $parent_refdes_path $sibling $sib_pin ]
    
            } else {
    
                set existing_pin [ concat $parent_refdes_path $pin_string ]
            }
    
            # Add local pin to proper net (and optionally create net for existing pin on-the-fly).
    
            add_connection_to_netlist $local_pin $existing_pin
        }
    
        refdes_path_pop
    }
    
    
    
    proc virtual { 'component' compname 'with' 'pins' pins 'consists' 'of' code } {
    
        make_proc $compname { refdes { connections {} } } {
    
            component_add_virtual [ concat [ refdes_path_get ] $refdes ]
    
            make_connections $connections $refdes
    
            refdes_path_push $refdes
    
            %CODE%
    
            refdes_path_pop
        } \
            %CODE%     $code
    }
    
    
    
    proc physical { 'component' compname 'with' 'pins' pins 'has' 'value' value 'and' 'footprint' footprint } {
    
        set_component_footprint $compname $footprint
    
        make_proc $compname { refdes { connections {} } } {
    
            component_add_physical [ concat [ refdes_path_get ] $refdes ] %VALUE% %FOOTPRINT%
    
            make_connections $connections $refdes
        } \
            %VALUE%     $value \
            %FOOTPRINT% $footprint
    }
    
    
    
    proc get_physical_component_properties refdes {
    
        global components
    
        if { ! ( $refdes in $components ) } { error "refdes '$refdes' not found in component-list (bug)" }
    
        dict get $components $refdes
    }
    
    
    
    proc stringified_refdes refdes { join $refdes _ }
    
    
    
    proc physical_components {} {
    
        global components
    
        set refdeses {}
    
        dict for { refdes comp_prop } $components {
    
            if { [ llength $comp_prop ] } {
    
                lappend refdeses $refdes
            }
        }
    
        return $refdeses
    }
    
    
    
    # $kicad_net_info = { $name, $nodes = { $refdes_string, $pin_nr }* }*
    
    proc kicad_net_info net {
    
        set net_info [ dict create name "" nodes {} ]
    
        foreach node $net {
    
            if { [ llength $node ] == 1 } {
    
                # Net-entry is a single word, so use it as the net-name.
    
                dict set net_info name $node
    
            } else {
    
                # Multi-word net-entry - could either be a physical or virtual component & pin.
    
                set refdes [ lrange $node 0 end-1 ]
    
                set comp_prop [ get_physical_component_properties $refdes ]
    
                if { [ llength $comp_prop ] } {
    
                    # This refdes & pin belong to a physical component, so include it in our info.
    
                    dict lappend net_info nodes [ dict create        \
                        refdes_string [ stringified_refdes $refdes ] \
                        pin_nr        [ lindex $node end ]           \
                    ]
                }
            }
        }
    
        return $net_info
    }
    
    
    
    proc write_kicad_cmplist fname {
    
        set fd [ open $fname w ]
    
        puts $fd "Cmp-Mod V01"
        puts $fd ""
    
        foreach refdes [ physical_components ] {
    
            set comp_prop [ get_physical_component_properties $refdes ]
    
            puts $fd "BeginCmp"
            puts $fd "Reference = [ stringified_refdes $refdes ];"
            puts $fd "IdModule  = [ dict get $comp_prop footprint ];"
            puts $fd "EndCmp"
            puts $fd ""
        }
    
        puts $fd "EndListe"
    
        close $fd
    }
    
    
    
    proc write_kicad_netlist fname {
    
        set fd [ open $fname w ]

        puts $fd "(export (version D)"
    
    
        
        puts $fd "(components"
    
        foreach r [ physical_components ] {
    
            puts -nonewline $fd "(comp "
            puts -nonewline $fd "(ref [ stringified_refdes $r ]) "
            puts $fd "(value [ dict get [ get_physical_component_properties $r ] value ]))"
        }
    
        puts $fd ")"
    
    
    
        puts $fd "(nets"
    
        set netcode 0
    
        upvar #0 nets nets
    
        foreach net $nets {
    
            set net_info [ kicad_net_info $net ]
    
            puts -nonewline $fd "(net "
            puts -nonewline $fd "(code [ incr netcode ]) "
            puts $fd "(name \"[ dict get $net_info name ]\")"
    
            foreach node [ dict get $net_info nodes ] {
    
                puts -nonewline $fd "(node "
                puts -nonewline $fd "(ref [ dict get $node refdes_string ]) "
                puts $fd "(pin [ dict get $node pin_nr        ]))"
            }
    
            puts $fd ")"
        }
    
        puts $fd ")"
        
        
    
        puts $fd ")"
    
        close $fd
    }

That's all!