| Message ID | 20180701110804.32415-1-afaerber@suse.de |
|---|---|
| Headers | show |
| Series | net: A socket API for LoRa | expand |
Hi Alan, 2018-08-03 22:02 GMT+08:00 Alan Cox <gnomes@lxorguk.ukuu.org.uk>: >> I'm not yet too deep into LoRaWAN, but from the AT command interfaces >> I've seen there's confirmed and unconfirmed transmission modes that with >> PF_LORAWAN might be mapped to SOCK_STREAM and SOCK_DGRAM. Or do you see >> a way of doing both on a single PF_LORA SOCK_LORAWAN socket? > > SOCK_STREAM is not a confirmed message, but a reliable data stream where > packet boundaries have no meaning. > > SOCK_SEQPACKET is a reliable ordered stream where message boundaries have > meaning. > > SOCK_RDM is reliable messaging where there is no ordering. > > The standard socket API has no concept of a single connection doing both > reliable and unreliable messages. Thanks for the useful information. LoRaWAN has 4 kinds of data messages: Unconfirmed data up/down, Confirmed data up/down. Unconfirmed data up/down can be mapped to SOCK_DGRAM. Confirmed data up/down can be mapped to SOCK_SEQPACKET. Because, there is the FCnt (frame counter) field in frame header for the order. Regards, Jian-Hong Pan >> Additionally I've been looking into socket options at PF_LORA dgram >> layer for some radio options, but discarded that again for lack of >> precedence. Basically I wondered whether we could allow to choose SF, > > I don't know if it'll stretch that far in the right directions but to the > extent you can re-use bits of the wifi API and it makes sense it would be > good to do so. > > For stuff bound to a specific socket you need to use the generic SOCK_ > stuff or you may indeed need some socket options at the PF_LORA level - > which is fine, and the whole point of setsockopt passing layers around. > > Some of the existing general stuff like priority is probably quite > useable. > >> bandwidth, etc. on socket level and then apply those settings before >> sending one packet rather than expecting a global netlink operation that >> affects all sockets for that interface. > > Alan
Am 08.08.2018 um 22:36 schrieb Alan Cox: > On Sun, 5 Aug 2018 02:11:25 +0200 > Andreas Färber <afaerber@suse.de> wrote: >> Am 03.07.2018 um 17:11 schrieb Jian-Hong Pan: >>> 2018-07-01 19:07 GMT+08:00 Andreas Färber <afaerber@suse.de>: >> LoRa radio channels being half-duplex, we'd need to stop receiving in >> ndo_start_xmit and re-start receiving in the TX interrupt handler AFAIU. >> Yes, it's ugly - one reason I haven't implemented RX in sx1276 yet. > > Why - the signal is still floating around in the air, you can't unhear it > at the antenna. Why what? :) > If a given piece of hardware needs to flip between RX > and TX mode then it should be handled by that driver. Yes, and we are talking about that concrete sx1276 driver here, whose chipset has a state machine that only allows either rx or tx and also has standby and sleep modes with differing levels of data retention. The sx1301 is a different beast (and driver) and may allow both. In any case, the ndo_start_xmit hook is in each device driver. But the recvmsg hook I mentioned is at protocol layer, which exactly is the layering problem I see. > (Some ancient ethernet cards do this btw.. they can't listen and transmit > at the same time) So when do they start receiving? The issue here was that my original description, which you appear to have cut, suggested a continuous listen mode, interrupted by transmit. Jian-Hong didn't like that, with reference to the LoRaWAN spec that supposedly asks for only being in receive mode when expecting a message, likely to save on battery. So the question is, could we cleanly implement receiving only when the user asks us to, or is that a no-go? >>> - We can have a pre-defined table according to LoRaWAN Regional Parameters. >>> - Device driver declares the hardware's capability, for example >>> frequency, TX power. And then registers as a LoRaWAN compatible >>> device. >> >> That sounds like a layering violation. We rather need to expose all >> these tunable parameters individually at the LoRa layer, allowing the >> LoRaWAN layer to configure them as it pleases. Not the other direction. >> That still leaves my above question 4) open of how to implement each. > > Wifi already has the general policy database for the various existing > protocols. Please take a look at the CRDA agent and how it hooks into > wireless.It might need to some tweaking but it would be odd to have > different configuration schemes for different wifi protocols. CRDA/wireless-regdb had been brought up and I've been pointed to nl80211 and nl802154, which I've tried to make sense of for my initial nllora. Admittedly with limited success, as that code is slightly complex. We also seemed to have consensus here that we _should_ be reusing wireless-regdb but would need to extend it 1.) with sub-GHz frequency bands and 2.) duty-cycle limits for some of those bands. No maintainer commented on that so far. Thus I am working in tiny steps on providing netlink-layer commands in nllora that can dispatch the individual radio settings to drivers, which then upper layers can instrument as needed. And making my very first steps with netlink here, it appeared as if each technology has its own enums of commands and attributes, so I don't see how to reuse anything from Wifi here apart from some design inspiration. >> The use case I have in mind is this: User A opens a LoRaWAN socket and >> using maclorawan sends a packet P1. Here the LoRaWAN Regional Parameters >> and LoRaWAN Sync Word need to be applied. >> User B then opens a pure LoRa socket and transmits a packet P1' with >> different Sync Word, SF, BW, CR, etc. >> Afterwards user A wants to send another packet P2 via LoRaWAN - this >> needs to use the same LoRaWAN settings as before, not those used for >> LoRa in between. Therefore I was thinking about socket-level options, as >> netlink operations would be device-wide, with undesired side-effects. > > Agreed > >> Obviously in that scenario not both users can receive at the same time. > > That's a hardware question. Imagine a software defined radio. If your > limitation wouldn't exist in a pure software defined radio then it's > almost certainly a device level detal. An SDR would not be using this sx1276 device driver, I imagine. In fact I would expect an SDR device not to be in drivers/net/lora/ at all but to live in drivers/net/sdr/ and to consume ETH_P_LORA etc. skbs and just do the right thing for them depending on their type... >> interface but cleanly distinguished as ETH_P_GFSK or something. >> For example, the Chistera Pi HAT has both an RFM95W and an RFM22 module. > > Agreed if you can flip per packet. The SX127x can - it might involve delays of course. The SX130x doesn't even need to flip for sending AFAICT, it's just metadata after the payload in the FIFO. >> The next question arising is how the user would create such an skb. Just >> like I was hesitant about PF_LORAWAN originally, I'd like to avoid >> polluting the PF_ number space for each of these. Maybe have one PF_FSK >> as equivalent to PF_LORA and then have either a socket option or >> sockaddr field to select the modulation variants? Not sure how exactly >> those others differ from each other, that's why I tried to postpone the >> FSK topic and to focus on LoRa first - b) below. >> >> At this point we could also argue whether we need a PF_LORA at all or >> rather just some generic PF_RADIO with lots of options stored in its >> sockaddr. > > What matters most is mux/demux. > > If you've got something listening to data but without the structure > needed to identify multiple listeners and split out the data meaningfully > to those listeners according to parts of the packet then you've got no > reason to make it a protocol just use SOCK_PACKET and if need be BPF. Sorry, that doesn't parse for me. SOCK_PACKET must be a protocol on some PF_ protocol family, no? Are you suggesting I use SOCK_PACKET instead of SOCK_DGRAM in what is now net/lora/dgram.c? Or are you saying there's some generic implementation that we can reuse and scratch mine? > The reason we have a socket layer not /dev/ethernet0 is that it's > meaningful to divide messages up into flows, and to partition those flows > securely amongst multiple consumers/generators. For me the distinction is that a /dev/whatever0 would seem more suited for a stream of data to read/write, whereas sockets give us a bounded skb for packets at device driver level. These PHYs all broadcast something over the antenna when sending, with any addressing of listeners or senders being optional and MAC-specific, apart from the LoRa/FSK SyncWord as well as the various frequency etc. settings that determine what the receiver listens for. None of these PHYs define any mechanism like EtherType through which to identify upper-layer protocols. So in a way, listening is always in a promiscuous mode, and I guess we would need to try to parse each incoming packet as e.g. a LoRaWAN packet and just give up if it's too short or checksums don't match. Only at the layer of LoRaWAN and competing proprietary or custom protocols can we split received packets out to individual listeners. Does that give us any further clues for the design discussion here? Regards, Andreas -- SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg)
Hi, On Thu, Aug 09, 2018 at 12:59:39PM +0100, Alan Cox wrote: > > Yes, and we are talking about that concrete sx1276 driver here, whose > > chipset has a state machine that only allows either rx or tx and also > > has standby and sleep modes with differing levels of data retention. > > It's a hardware limit, it should never influence the protocol stack > itself just the driver. Linux always tries to design to optimize the > non-crappy case. In the long term that works out best because hardware > improves and you don't want to be tied to an old limit. > > > > (Some ancient ethernet cards do this btw.. they can't listen and transmit > > > at the same time) > > > > So when do they start receiving? > > When they are not transmitting. The transmit path switches modes and when > the frame send is done it goes back to receiving. As old ethernet was > also half duplex that worked. > We do the same at some IEEE 802.15.4 transceivers. A transceiver has _one_ framebuffer only for tx and rx. Another one has two framebuffer separated tx and rx, but is half duplex. There is a little performance tweak in separated framebuffers that you can fill up the tx framebuffer while the transceiver receives the frame (completely independet from any bus communicaten/linux handling). > > The issue here was that my original description, which you appear to > > have cut, suggested a continuous listen mode, interrupted by transmit. > > I don't think I cut it but if so I didn't mean to and your approach is > the one I agree with. > ... > > There is a heirarchy. Let me us IP for an example > > (historically it was SOCK_PACKET nowdays PF_PACKET - the layering got > sorted better) > > PF_PACKET SOCK_RAW ETH_P_ALL > > Everything on that device minus some things like hardware pre-ambles > > PF_PACKET SOCK_RAW ETH_P_SOMETHING > > Everything on that device that has the underlying protocol (and the > protocol might not be in the packet but a property of the interface > because it only does that format - simple example SLIP is IP packets over > a serial link a SLIP interface is IP, not because there is anything > saying it is but because that is *all* it can be) > > You get the two above for free. PF_PACKET is built into the stack so > providing you label packets with the ETH_P_xxx you have for Lora, you can > use PF_PACKET interfaces to dump them and write raw packets at the kernel > layer. > In 802.15.4: We recommend nowadays to use PF_PACKET raw sockets for construct L2 frames in userspace. We use that mostly to connect some user space stacks to make some interop testing without hardware being involved. For DGRAM sockets, due lack of UAPI limitations of sockaddr_t we have our own implementation. DGRAM in PF_PACKET use some limitated feature to "just send something" to an unique address scheme... but some users need more access because 802.15.4 address scheme is complex. > PF_INET SOCK_RAW I think LoRa should look into the 6lowpan subsystem of the Linux kernel for that. 6lowpan is known as some "IPv6-over-foo" adaptations. Mostly you just need to implement some mapping from L2 address to SLAAC IPv6 address pattern only. I see there is a draft at 6lo wg [0] for that which is expired 2016 (But I would not care about that). At the end you have a master IP capable interface and your slave is your L2 interface. The 6lowpan interface is a RAW IP interface and do a protocol translation in the background. On L2 interface you will see L2 + 6LoWPAN + $IP_UPPER_LAYER. Current benefits are more compression, but there exists also some ndisc optimizations for low power networks which we don't support right now. - Alex [0] https://www.ietf.org/archive/id/draft-vilajosana-6lpwa-lora-hc-01.txt
On 7/1/18 1:08 PM, Andreas Färber wrote: > The IMST WiMOD uses a SLIP based binary UART protocol. Two separate > firmwares are available. By default it ships with a LoRaWAN firmware. > The alternative firmware is a custom P2P addressing mode based on LoRa. > > Cc: Jon Ortego <Jon.Ortego@imst.de> > Signed-off-by: Andreas Färber <afaerber@suse.de> > --- > drivers/net/lora/Kconfig | 8 + > drivers/net/lora/Makefile | 3 + > drivers/net/lora/wimod.c | 597 ++++++++++++++++++++++++++++++++++++++++++++++ > 3 files changed, 608 insertions(+) > create mode 100644 drivers/net/lora/wimod.c > > diff --git a/drivers/net/lora/Kconfig b/drivers/net/lora/Kconfig > index 940bd2cbe106..2e05caef8645 100644 > --- a/drivers/net/lora/Kconfig > +++ b/drivers/net/lora/Kconfig > @@ -31,6 +31,14 @@ config LORA_SX1276 > help > Semtech SX1272/1276/1278 > > +config LORA_WIMOD > + tristate "IMST WiMOD driver" scripts/checkpatch.pl throws this warning: WARNING: please write a paragraph that describes the config symbol fully IMST has multiple products related to "WiMOD": * WiMOD iC880A * WiMOD module iM871A * WSA01-iM880B - WiMOD Shield for Arduino * iM880B-L - Long Range Radio Module And IMST is not very consistent about what is called "WiMOD". So this leaves me clueless concerning what this Kconfig option is about. Please, provide a description. There are dozens of warnings given by scripts/checkpatch. Please, have a look at them. Best regards Heinrich
Hello, LoRa is a long-range, low-power wireless technology by Semtech. Unlike other LPWAN technologies, users don't need to rely on infrastructure providers and SIM cards and expensive subscription plans, they can set up their own gateways. Modules, adapters and evaluation boards are available from a large number of vendors. Many vendors also make available Open Source software examples on GitHub. But when taking a closer look, many of them combine licenses in ways that are not redistributable. My reports have remained without response or solution. https://github.com/ernstdevreede/lmic_pi/issues/2 https://github.com/Snootlab/lmic_chisterapi/issues/2 https://github.com/Snootlab/lora_chisterapi/issues/2 Another issue was that most such projects around the Raspberry Pi make use of spidev to communicate with the Semtech chipsets from userspace. The Linux spi maintainers have chosen to greet any such users of spidev with a friendly WARN_ON(), preferring in-kernel spi drivers and white-listing individual devices only. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/spi/spidev.c?h=v4.17#n722 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/spi/spidev.c?h=v4.17#n667 Also I don't quite see the point in having userspace probe what SPI devices are connected to a generic spidev driver when we have an easy Device Tree hardware description on arm/arm64 that could give us that info. I raised the topic during Q&A of a FOSDEM 2017 talk (cut off at the end of the video) but unfortunately found no one to collaborate on this. https://archive.fosdem.org/2017/schedule/event/lorawan/ Instead of porting from wiringPi to a differently licensed GPIO library and dealing with seemingly unmaintained LoRaWAN code dumps, I started a spi kernel driver for SX1276 back in 2016. But obviously a kernel driver isn't too helpful without a userspace API to send and receive packets. This patchset, updated from 2017 and extended, is implementing kernel drivers for various LoRa chipsets and modules. As API I'm proposing a PF_LORA socket implementation. Why? LoRa uses data packets with headers and checksums and differing MTUs and multiple protocols layered on top of it. Apart from simple headers for addressing used by RadioHead library and IMST's LoRa P2P protocol, the main use case (not implemented in this patchset) is expected to be LoRaWAN. And LoRaWAN has competing proprietary protocols, such as Link Labs' Symphony Link or GlobalSat M.O.S.T. or RadioShuttle, that might at some point want to adopt a standard API for their implementations, too. Ready-made LoRa hardware modules come in three flavors, a) with SPI access to the underlying Semtech chipsets, needing a software implementation of e.g. LoRaWAN protocol stack (i.e., a soft MAC), b) with a custom, often UART based interface and a pre-certified LoRaWAN protocol stack already integrated (i.e., hard/full MAC), and c) with a microcontroller that serves not only for the protocol stack but also as application processor, not offering a ready-made interface. This patchset focuses on option a). An SX1276 based LoRaWAN stack appeared to be the project of Jian-Hong Pan and is not included here. This patchset also includes drivers for b), from text based AT commands to a binary SLIP based HCI protocol. Hardware examples for c) are Murata CMWX1ZZABZ-078 and RAK813. This patchset is clearly not ready for merging, but is being submitted for discussion, as requested by Jiri, in particular of the design choices: 1) PF_LORA/AF_LORA and associated identifiers are proposed to represent this technology. While for an SX1276 - case a) above - it might work to layer LoRaWAN as a protocol option for PF_LORA and add LoRaWAN address fields to the union in my sockaddr_lora, how would that work for devices that only support LoRaWAN but not pure LoRa? Do we need both AF_LORA and AF_LORAWAN, or just a separate ETH_P_LORAWAN or ARPHRD_LORAWAN? 2) PF_LORA is used with SOCK_DGRAM here. The assumption is that RAW mode would be DGRAM plus preamble plus optional checksum. 3) Only the transmit path is partially implemented already. The assumption is that the devices should go into receive mode by default and only interrupt that when asked to transmit. 4) Some hardware settings need to be supplied externally, such as the radio frequency for some modules, but many others can be runtime-configured, such as Spreading Factor, Bandwidth, Sync Word, or which antenna to use. What settings should be implemented as socket option vs. netlink layer vs. ioctl vs. sysfs? What are the criteria to apply? 5) Many of the modules support multiple modes, such as LoRa, LoRaWAN and FSK. Lacking a LoRaWAN implementation, I am currently switching them into LoRa mode at probe time wherever possible. How do we deal with that properly? a) Is there any precedence from the Wifi world for dynamically selecting between our own trusted Open Source implementation vs. hardware/firmware accelerated and/or certified implementations? b) Would a proof of concept for FSK (non-LoRa) modes be required for merging any LoRa driver for chipsets that support both? Or is there any facility or design guidelines that would allow us to focus on LoRa and LoRaWAN and leave non-LoRa radio modes to later contributors? As evident by the many questions, this is my first deep dive into the Linux net subsystem. It's also my first experiments with the new serdev subsystem, so in particular the receive paths will need some review and optimizations. This patchset was developed and tested mainly as KMP, originally at https://github.com/afaerber/lora-modules. It was recently transformed into a linux-next based tree, still mostly tested on our openSUSE Tumbleweed kernel with a differing AF_LORA value below current AF_MAX limit. Some corresponding Device Tree Overlays have been collected here: https://github.com/afaerber/dt-overlays Only European models for 868 MHz and 433 MHz could be tested when available. Thanks to all companies and people that have supported this project so far. Have a lot of fun! Cheers, Andreas Cc: Jian-Hong Pan <starnight@g.ncu.edu.tw> Cc: Jiri Pirko <jiri@resnulli.us> Cc: Marcel Holtmann <marcel@holtmann.org> Cc: "David S. Miller" <davem@davemloft.net> Cc: Matthias Brugger <mbrugger@suse.com> Cc: Konstantin Böhm <konstantin.boehm@ancud.de> Cc: Jan Jongboom <jan.jongboom@arm.com> Cc: Janus Piwek <jpiwek@arroweurope.com> Cc: Michael Röder <michael.roeder@avnet.eu> Cc: Dollar Chen (陳義元) <dollar.chen@wtmec.com> Cc: Ken Yu (禹凯) <ken.yu@rakwireless.com> Cc: Jon Ortego <Jon.Ortego@imst.de> Cc: contact@snootlab.com Cc: Ben Whitten <ben.whitten@lairdtech.com> Cc: Brian Ray <brian.ray@link-labs.com> Cc: lora@globalsat.com.tw Cc: lora@radioshuttle.de Cc: Alexander Graf <agraf@suse.de> Cc: Michal Kubeček <mkubecek@suse.cz> Cc: Rob Herring <robh@kernel.org> Cc: devicetree@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org Cc: Steve deRosier <derosier@gmail.com> Cc: Mark Brown <broonie@kernel.org> Cc: linux-spi@vger.kernel.org Andreas Färber (15): net: Reserve protocol numbers for LoRa net: lora: Define sockaddr_lora net: lora: Add protocol numbers net: Add lora subsystem HACK: net: lora: Deal with .poll_mask in 4.18-rc2 net: lora: Prepare for device drivers net: lora: Add Semtech SX1276 net: lora: sx1276: Add debugfs net: lora: Prepare EUI helpers net: lora: Add Microchip RN2483 net: lora: Add IMST WiMOD net: lora: Add USI WM-SG-SM-42 net: lora: Prepare RAK RAK811 net: lora: Prepare Semtech SX1257 net: lora: Add Semtech SX1301 drivers/net/Makefile | 1 + drivers/net/lora/Kconfig | 72 ++++ drivers/net/lora/Makefile | 32 ++ drivers/net/lora/dev.c | 125 ++++++ drivers/net/lora/rak811.c | 219 +++++++++++ drivers/net/lora/rn2483.c | 344 +++++++++++++++++ drivers/net/lora/rn2483.h | 40 ++ drivers/net/lora/rn2483_cmd.c | 130 +++++++ drivers/net/lora/sx1257.c | 96 +++++ drivers/net/lora/sx1276.c | 740 ++++++++++++++++++++++++++++++++++++ drivers/net/lora/sx1301.c | 446 ++++++++++++++++++++++ drivers/net/lora/usi.c | 411 ++++++++++++++++++++ drivers/net/lora/wimod.c | 597 +++++++++++++++++++++++++++++ include/linux/lora/dev.h | 44 +++ include/linux/lora/skb.h | 29 ++ include/linux/socket.h | 4 +- include/uapi/linux/if_arp.h | 1 + include/uapi/linux/if_ether.h | 1 + include/uapi/linux/lora.h | 24 ++ net/Kconfig | 1 + net/Makefile | 1 + net/lora/Kconfig | 15 + net/lora/Makefile | 8 + net/lora/af_lora.c | 152 ++++++++ net/lora/af_lora.h | 13 + net/lora/dgram.c | 297 +++++++++++++++ security/selinux/hooks.c | 4 +- security/selinux/include/classmap.h | 4 +- 28 files changed, 3848 insertions(+), 3 deletions(-) create mode 100644 drivers/net/lora/Kconfig create mode 100644 drivers/net/lora/Makefile create mode 100644 drivers/net/lora/dev.c create mode 100644 drivers/net/lora/rak811.c create mode 100644 drivers/net/lora/rn2483.c create mode 100644 drivers/net/lora/rn2483.h create mode 100644 drivers/net/lora/rn2483_cmd.c create mode 100644 drivers/net/lora/sx1257.c create mode 100644 drivers/net/lora/sx1276.c create mode 100644 drivers/net/lora/sx1301.c create mode 100644 drivers/net/lora/usi.c create mode 100644 drivers/net/lora/wimod.c create mode 100644 include/linux/lora/dev.h create mode 100644 include/linux/lora/skb.h create mode 100644 include/uapi/linux/lora.h create mode 100644 net/lora/Kconfig create mode 100644 net/lora/Makefile create mode 100644 net/lora/af_lora.c create mode 100644 net/lora/af_lora.h create mode 100644 net/lora/dgram.c -- 2.16.4