mbox series

[v2,0/7] tty: add flag to suppress ready signalling on open

Message ID 20201202113942.27024-1-johan@kernel.org
Headers show
Series tty: add flag to suppress ready signalling on open | expand

Message

Johan Hovold Dec. 2, 2020, 11:39 a.m. UTC
This series adds a new NORDY port flag to suppress raising the
modem-control lines on open to signal DTE readiness.

This can be used to implement a NORDY termios control flag to complement
HUPCL, which controls lowering of the modem-control lines on final
close.

Initially drivers can export the flag through sysfs, which also allows
control over the lines on first open. Such an interface is implemented
for serial core and USB serial.

The motivation for this is to allow for applications where the DTR and
RTS lines are used for non-standard purposes (e.g. generating power-on
and reset pulses) to open the port without undesirable side effects.

The final patches enables this flag by default for such a USB serial
device.

Other examples include HAM-radio devices where DTR and RTS is used for
push-to-talk and continuous-wave control and various Arduino boards
which are reset on open unless a jumper is physically removed.

Greg, are you ok we me taking this through my tree? I'm planning on some
follow ups to the ftdi driver and the tty/serial changes are fairly
self-contained.

Also let me know if you prefer to hold this off for 5.12. The change is
minimal, self-contained and low-risk, but it is a new interface and late
in the release cycle as Andy pointed out.

Johan


Changes in v2
 - use assign_bit() in port-flag accessors (Jiri)
 - use const parameters in port-flag accessors (Jiri)
 - use kstrtobool() in attribute store (Andy, Greg)
 - fix "used" typo in commit messages (Andy)
 - use bool constant with port-flag accessor in ftdi quirk (Jiri)
 - add Mychaela's Reviewed-by tag to patch 3/7
 - mention a few more example applications in the cover letter

v1
 - https://lore.kernel.org/r/20201130153742.9163-1-johan@kernel.org


Johan Hovold (5):
  tty: use assign_bit() in port-flag accessors
  tty: use const parameters in port-flag accessors
  tty: add port flag to suppress ready signalling on open
  serial: core: add sysfs attribute to suppress ready signalling on open
  USB: serial: add sysfs attribute to suppress ready signalling on open

Mychaela N. Falconia (2):
  USB: serial: ftdi_sio: pass port to quirk port_probe functions
  USB: serial: ftdi_sio: add support for FreeCalypso DUART28C adapter

 Documentation/ABI/testing/sysfs-tty |  7 ++++
 drivers/tty/serial/serial_core.c    | 26 ++++++++++++++
 drivers/tty/tty_port.c              |  2 +-
 drivers/usb/serial/bus.c            | 35 +++++++++++++++++--
 drivers/usb/serial/ftdi_sio.c       | 42 ++++++++++++++++++-----
 drivers/usb/serial/ftdi_sio_ids.h   |  1 +
 include/linux/tty.h                 | 53 +++++++++++++----------------
 7 files changed, 125 insertions(+), 41 deletions(-)

Comments

Mychaela Falconia Dec. 2, 2020, 6:07 p.m. UTC | #1
On 12/2/20, Johan Hovold <johan@kernel.org> wrote:
> Also let me know if you prefer to hold this off for 5.12. The change is
> minimal, self-contained and low-risk, but it is a new interface and late
> in the release cycle as Andy pointed out.

Hold off for 5.12? Did you perhaps mean 5.11? I understand how this
change may be too late for 5.10, but surely it can go into 5.11 merge
window, why hold off for 5.12?

M~
Johan Hovold Dec. 3, 2020, 9:21 a.m. UTC | #2
On Wed, Dec 02, 2020 at 10:07:58AM -0800, Mychaela Falconia wrote:
> On 12/2/20, Johan Hovold <johan@kernel.org> wrote:
> > Also let me know if you prefer to hold this off for 5.12. The change is
> > minimal, self-contained and low-risk, but it is a new interface and late
> > in the release cycle as Andy pointed out.
> 
> Hold off for 5.12? Did you perhaps mean 5.11? I understand how this
> change may be too late for 5.10, but surely it can go into 5.11 merge
> window, why hold off for 5.12?

No, I meant 5.12, even if I still think this could go into 5.11.

Generally new features should brew a bit in linux-next. It's mostly bots
testing linux-next, and they wouldn't be able to catch anything beyond
potential build issues as this feature is off by default anyway. So I
don't think that needs to be an issue in this case.

But we don't have deadlines and if, say, for one reason or another Greg
doesn't have time to review this in the next couple of weeks, we'll just
hold it off.

Johan
Greg Kroah-Hartman Dec. 9, 2020, 2:30 p.m. UTC | #3
On Wed, Dec 02, 2020 at 12:39:35PM +0100, Johan Hovold wrote:
> This series adds a new NORDY port flag to suppress raising the
> modem-control lines on open to signal DTE readiness.
> 
> This can be used to implement a NORDY termios control flag to complement
> HUPCL, which controls lowering of the modem-control lines on final
> close.
> 
> Initially drivers can export the flag through sysfs, which also allows
> control over the lines on first open. Such an interface is implemented
> for serial core and USB serial.
> 
> The motivation for this is to allow for applications where the DTR and
> RTS lines are used for non-standard purposes (e.g. generating power-on
> and reset pulses) to open the port without undesirable side effects.
> 
> The final patches enables this flag by default for such a USB serial
> device.
> 
> Other examples include HAM-radio devices where DTR and RTS is used for
> push-to-talk and continuous-wave control and various Arduino boards
> which are reset on open unless a jumper is physically removed.
> 
> Greg, are you ok we me taking this through my tree? I'm planning on some
> follow ups to the ftdi driver and the tty/serial changes are fairly
> self-contained.
> 
> Also let me know if you prefer to hold this off for 5.12. The change is
> minimal, self-contained and low-risk, but it is a new interface and late
> in the release cycle as Andy pointed out.

I took the first 2 patches now, that was easy :)

I think we need more review for the rest of the series.  This does
change the way serial ports work in a non-traditional way (i.e. using
sysfs instead of terminal settings).  So I want to get a bunch of people
to agree that this is ok to do things this way now before taking this
new user-visible api.

thanks,

greg k-h
Johan Hovold Dec. 9, 2020, 3:10 p.m. UTC | #4
On Wed, Dec 09, 2020 at 03:30:33PM +0100, Greg Kroah-Hartman wrote:
> On Wed, Dec 02, 2020 at 12:39:35PM +0100, Johan Hovold wrote:
> > This series adds a new NORDY port flag to suppress raising the
> > modem-control lines on open to signal DTE readiness.
> > 
> > This can be used to implement a NORDY termios control flag to complement
> > HUPCL, which controls lowering of the modem-control lines on final
> > close.
> > 
> > Initially drivers can export the flag through sysfs, which also allows
> > control over the lines on first open. Such an interface is implemented
> > for serial core and USB serial.

> > Also let me know if you prefer to hold this off for 5.12. The change is
> > minimal, self-contained and low-risk, but it is a new interface and late
> > in the release cycle as Andy pointed out.
> 
> I took the first 2 patches now, that was easy :)
> 
> I think we need more review for the rest of the series.  This does
> change the way serial ports work in a non-traditional way (i.e. using
> sysfs instead of terminal settings).  So I want to get a bunch of people
> to agree that this is ok to do things this way now before taking this
> new user-visible api.

Sounds good, thanks.

Johan
Mychaela Falconia Dec. 9, 2020, 10:49 p.m. UTC | #5
Greg K-H wrote:

> I think we need more review for the rest of the series.  This does
> change the way serial ports work in a non-traditional way (i.e. using
> sysfs instead of terminal settings).

But the problem is that the current status quo is fundamentally broken
for those hardware devices in which DTR and/or RTS have been repurposed
for something other than modem and flow control.  Right now whenever a
"cold" (never previously opened) serial port is opened for the first
time, that open action immediately and unstoppably asserts both DTR
and RTS hardware outputs, without giving userspace any opportunity to
say "no, please don't do it".  Yes, this behaviour is codified in a
bunch of standards that ultimately trace back to 1970s Original UNIX,
but just because it is a standard does not make it right - this
Unix/POSIX/Linux "standard" serial port behaviour is a bug, not a
feature.

All operating systems are fundamentally middlemen between applications
and hardware.  If custom application A is specifically written to work
with custom hardware H, it is the operating system's job to allow A to
operate on H without unreasonable interference, NOT to tell A and H
that they should change their hw and/or app design to fit a given OS
worldview.

Furthermore, hardware engineers who design custom gadgets have a
natural right to reduce hardware cost by employing designs that are
simple and elegant in hw terms - they should NOT be forced to make hw
designs more complicated, more expensive, or less convenient for end
users in order to please an OS.  To put it another way, OSes and OS
maintainers need to see themselves as working for end users and their
hardware and applications, not the other way around.

Here is my situation: I needed a little hardware adapter that goes
from USB to one UART with full modem and flow control signals (8-wire
UART), one more UART with only data leads and no modem or flow control
(2-wire UART), and two non-serial control signals that would allow a
connected host computer to simulate PWON and RESET pushbutton presses.
The simplest and most elegant hardware solution was to use an FT2232x
chip from FTDI (either FT2232D or FT2232H, I chose the D version
because I didn't need USB high speed), wire up one channel for the
8-wire UART, wire up the other channel for the 2-wire UART, and then
repurpose the otherwise unused Channel B RTS & DTR outputs to drive
PWON and RESET pushbutton signals via an open drain buffer IC.

The entire hardware solution for driving my PWON and RESET signals
consists of a single buffer IC that measures 1.00x1.45 mm - an epitome
of hardware simplicity and elegance, I would argue.  If my target OS
was Windows, I wouldn't be having all these problems: under Windows
when you open a "cold" serial port (USB-serial converter just plugged
in, not previously opened) for the first time, that first-open action
does NOT jerk DTR or RTS modem control signals, instead they are left
alone - and the application program can then control and configure how
they should function subsequently.  But Linux is being a bad boy: when
the serial port is opened, DTR and RTS are immediately asserted, and
there is no way to prevent it other than by applying a local patch to
the kernel on your system.

I realize that changing the default first-open behaviour to leave DTR
and RTS alone instead of asserting them would not be acceptable.
There are lots of devices that require DTR and/or RTS to be asserted
for serial communication to work, and because the standard (however
poorly thought through) behaviour has always been to auto-assert these
signals on open, most standard serial applications rely on this
automatic assertion - almost no one does an explicit TIOCMBIS of DTR
and RTS after open.

Thinking as a hardware engineer, to me the most natural solution to
this Linux problem is to use a custom USB ID upon which a driver quirk
is then conditionalized.  All major USB-serial chips feature either a
built-in EEPROM or OTP configuration memory, or a provision for
connecting a board-level EEPROM.  One of the parameters set in this
hardware config EEPROM is the USB VID:PID which the USB-serial chip
will present itself as.  Furthermore, most USB-serial chip vendors (at
least FTDI and Silabs for sure) allow their customers (designers of
custom hw containing their chips) to obtain a custom PID out of that
chip vendor's USB VID, making it very very easy for custom USB-serial
boards to have custom USB IDs.

So I program the EEPROM on my custom DUART28 board with the custom USB
ID which FTDI officially allocated to me (thus this ID code absolutely
uniquely identifies my specific hardware and no other), and I prepare
a patch for the ftdi_sio driver which I thought would be
non-controversial: adding support for my new FT2232D-based device by
adding its custom USB ID to the ID code table, along with a quirk that
suppresses automatic assertion of DTR & RTS on open for the Channel B
ttyUSB device.  I thought my patch would be non-controversial: it is
simply adding support for a new hardware device which was not
previously supported by Linux at all, same as if I were adding an
entirely new driver, and has ZERO impact on any other hardware, any
other applications, any other users, any other anything - just support
for a new hw device with a special quirk and nothing more.

But Johan objects to simply merging my ftdi_sio driver patch by itself
because it is not a general solution, because it covers only one
specific hw device.  Johan proposed a more generalized solution that
would still satisfy me and the end users of my hardware because in the
end it still includes recognition of my custom USB ID, setting the
needed quirk flag upon seeing this ID - but now Johan's generalized
patch series is not acceptable either?  So what gives?

At the end of the day we have a situation where the creators of
Original UNIX have made a design mistake back in the 1970s, and this
design mistake persists to this day in all current Unix-style OSes
including Linux.  The design mistake in question is unconditionally
asserting DTR and RTS upon the first open of a previously cold serial
port.  This behaviour is harmless for most devices in which DTR and
RTS retain their standard RS-232 functions, but it is a killer for
those hw devices in which either or both signals have been repurposed
for something very different, some other function that does not
tolerate unwanted signal assertion.

Unix/POSIX/Linux OSes are at fault here, I argue, NOT hardware
engineers who build gadgets in which these signals are repurposed.
The physical hardware behaviour of a classic old-fashioned RS-232 DTE
port is that when the hardware powers up, the output voltage on DTR
and RTS lines goes from zero to -12 V, indicating negated state - it
never goes toward +12 V corresponding to asserted state, until and
unless software takes action to raise those signals.  Same deal with
modern USB-serial chips: when USB VBUS power is applied, DTR and RTS
outputs immediately go CMOS high (still corresponding to negated
state), or they remain tristated until enumeration, in which case
pull-up resistors solve the problem - but they *never* go into the
state of actively driving CMOS low (overriding pull-up resistors)
until and unless software sends the necessary USB control pipe
command.

Hardware engineers have every right to design gadgets that take
actions upon asserted DTR/RTS (CMOS low or RS-232 +12V), actions that
should not occur when not explicitly requested, and we the hardware
engineers feel justified in this stance because the state of CMOS low
or RS-232 +12V on DTR and RTS control lines never occurs in the
hardware on its own, this state only occurs upon software command -
thus we the hardware engineers have the right to ask OS maintainers to
provide a way to NOT send that software command until and unless the
user's application explicitly requests it.

If this design flaw had been noticed back in the 1970s and addressed
in some way, we wouldn't be having this discussion.  But here we are
almost a half-century later, and the problem is still unresolved.  So
how do we solve this problem now, after having ignored it for 45 y?
Simply changing the default action upon first open of a previously
cold serial port to not automatically assert DTR & RTS won't work:
plenty of devices (classic modems etc) require DTR and/or RTS to be
asserted for communication to work, and plenty of applications rely on
automatic assertion of these signals on the first open of a previously
cold serial port.  Adding configurable termios flags etc won't help
either: one needs to open a port in order to do ioctls on it, and if
that first open still raises DTR & RTS, then users of custom hw are
still screwed.

Thus we need some out-of-band mechanism.  There are only 3
possibilities:

1) Reusing/abusing some open flag like O_DIRECT, perhaps only for the
very first open, and then setting a termios or setserial flag to
change behaviour on subsequent opens;

2) A sysfs attribute like in Johan's proposed patch series;

3) ID-code-specific quirks in USB-serial chip drivers.

At this point an important question needs to be asked.  Johan mentioned
that the issue I am dealing with (needing some way to suppress unwanted
auto-assertion of DTR/RTS) has come up in the past in the context of
ham radio devices.  Does Johan or anyone else know what their physical
form factor and serial interface was/is?  Was it true old-fashioned
RS-232 such that the device could be connected to "any" serial port,
or was it a custom USB-interfaced board with an integrated USB-serial
chip?

If every non-standard serial port device currently in existence that
does not tolerate automatic assertion of DTR/RTS and requires the
changes we discussing takes the physical form of a custom circuit
board with a USB connector and a built-in USB-serial chip, with all
UART signals staying in the CMOS (or more likely LVCMOS) logic voltage
level domain without ever becoming old-fashioned RS-232, then the
whole argument about supporting "any" serial port becomes moot.  In
this case the simplest and least invasive solution is to assign custom
USB IDs to all such non-standard devices by way of their respective
EEPROMs, and support these custom USB ID devices with driver quirks -
no need for sysfs, no need for O_DIRECT, no need to touch any kernel
code at all outside of individual USB-serial chip drivers.

But if there exist some custom hw devices out there that are in the
same predicament as my DUART28 adapter, but are different in that they
are classic old-fashioned RS-232 rather than integrated USB-serial,
with no place to assign a custom USB ID, *then* we need a non-USB-ID-
dependent solution such as Johan's sysfs attribute or O_DIRECT.

> So I want to get a bunch of people
> to agree that this is ok to do things this way now before taking this
> new user-visible api.

So where and how are we going to find that bunch of people?  What
happens if no one gives a damn one way or another?  Neither I nor Johan
are proposing any change whatsoever to any existing behaviour or any
existing interfaces - simply providing a new option, such that if you
don't care about it and don't set it, absolutely nothing changes and
nothing is impacted in any way whatsoever.

If the concern is with the new sysfs interface or the proposed O_DIRECT
alternative, how about deferring those while allowing specific USB ID
support to go in first?  Right now there already exists at least one
piece of hardware actively supported by its manufacturer (my gadget)
that has a custom USB ID and requires the quirk - what is wrong with
adding support for this existing specific hw?  How about merging
Johan's patch that defines the NORDY flag in tty_port, merging the
ftdi_sio driver patch setting this flag for my custom USB ID, allowing
other hardware engineers in the same boat to submit similar quirk
patches for their affected custom hw with custom USB IDs, while
deferring the sysfs patches until there is a more pressing need for
quirky devices that have no custom USB IDs?

Sincerely,
Mychaela
Greg Kroah-Hartman Dec. 10, 2020, 9:06 a.m. UTC | #6
On Wed, Dec 09, 2020 at 02:49:45PM -0800, Mychaela Falconia wrote:
> Greg K-H wrote:

> 

> > I think we need more review for the rest of the series.  This does

> > change the way serial ports work in a non-traditional way (i.e. using

> > sysfs instead of terminal settings).

> 

> But the problem is that the current status quo is fundamentally broken

> for those hardware devices in which DTR and/or RTS have been repurposed

> for something other than modem and flow control.  Right now whenever a

> "cold" (never previously opened) serial port is opened for the first

> time, that open action immediately and unstoppably asserts both DTR

> and RTS hardware outputs, without giving userspace any opportunity to

> say "no, please don't do it".  Yes, this behaviour is codified in a

> bunch of standards that ultimately trace back to 1970s Original UNIX,

> but just because it is a standard does not make it right - this

> Unix/POSIX/Linux "standard" serial port behaviour is a bug, not a

> feature.


Thanks for the long response, but I think you have to realize that
creating a new api for something that has been "how things work" since
the 1970's should not be taken lightly.  No matter if it was a bug or
not, changing user-visable behavior is not a trivial thing.  What we
come up with here has to stand the test of time of being able to be
supported properly for the next 40+ years.

thanks,

greg k-h
Maarten Brock Dec. 10, 2020, 10:41 a.m. UTC | #7
Hello Mychaela,

On 2020-12-09 23:49, Mychaela Falconia wrote:
> Greg K-H wrote:
> 
>> I think we need more review for the rest of the series.  This does
>> change the way serial ports work in a non-traditional way (i.e. using
>> sysfs instead of terminal settings).
> 
> But the problem is that the current status quo is fundamentally broken
> for those hardware devices in which DTR and/or RTS have been repurposed
> for something other than modem and flow control.  Right now whenever a
> "cold" (never previously opened) serial port is opened for the first
> time, that open action immediately and unstoppably asserts both DTR
> and RTS hardware outputs, without giving userspace any opportunity to
> say "no, please don't do it".  Yes, this behaviour is codified in a
> bunch of standards that ultimately trace back to 1970s Original UNIX,
> but just because it is a standard does not make it right - this
> Unix/POSIX/Linux "standard" serial port behaviour is a bug, not a
> feature.

I agree. And an application not configuring the required handshakes, but
still relying on them is an equal bug.

> But if there exist some custom hw devices out there that are in the
> same predicament as my DUART28 adapter, but are different in that they
> are classic old-fashioned RS-232 rather than integrated USB-serial,
> with no place to assign a custom USB ID, *then* we need a non-USB-ID-
> dependent solution such as Johan's sysfs attribute or O_DIRECT.

Any device with a classic old-fashioned RS-232 has probably already
solved this in another way or is accepted as not working on Linux.

And then there is also the device tree (overlay?) through which a quirk
like this can be communicated to the kernel driver. Not sure if this
could help for a plug-and-play device like on USB.

>> So I want to get a bunch of people
>> to agree that this is ok to do things this way now before taking this
>> new user-visible api.

Personally, I would prefer the VID:PID to enforce the quirk and an
O_DIRECT (or other) flag used on open() as general backup plan. To
me a sysfs solution seems illogical.

> If the concern is with the new sysfs interface or the proposed O_DIRECT
> alternative, how about deferring those while allowing specific USB ID
> support to go in first?  Right now there already exists at least one
> piece of hardware actively supported by its manufacturer (my gadget)
> that has a custom USB ID and requires the quirk - what is wrong with
> adding support for this existing specific hw?  How about merging
> Johan's patch that defines the NORDY flag in tty_port, merging the
> ftdi_sio driver patch setting this flag for my custom USB ID, allowing
> other hardware engineers in the same boat to submit similar quirk
> patches for their affected custom hw with custom USB IDs, while
> deferring the sysfs patches until there is a more pressing need for
> quirky devices that have no custom USB IDs?
> 
> Sincerely,
> Mychaela

Again, I agree.

Maarten
Greg Kroah-Hartman Dec. 10, 2020, 10:50 a.m. UTC | #8
On Thu, Dec 10, 2020 at 11:41:24AM +0100, Maarten Brock wrote:
> Hello Mychaela,
> 
> On 2020-12-09 23:49, Mychaela Falconia wrote:
> > Greg K-H wrote:
> > 
> > > I think we need more review for the rest of the series.  This does
> > > change the way serial ports work in a non-traditional way (i.e. using
> > > sysfs instead of terminal settings).
> > 
> > But the problem is that the current status quo is fundamentally broken
> > for those hardware devices in which DTR and/or RTS have been repurposed
> > for something other than modem and flow control.  Right now whenever a
> > "cold" (never previously opened) serial port is opened for the first
> > time, that open action immediately and unstoppably asserts both DTR
> > and RTS hardware outputs, without giving userspace any opportunity to
> > say "no, please don't do it".  Yes, this behaviour is codified in a
> > bunch of standards that ultimately trace back to 1970s Original UNIX,
> > but just because it is a standard does not make it right - this
> > Unix/POSIX/Linux "standard" serial port behaviour is a bug, not a
> > feature.
> 
> I agree. And an application not configuring the required handshakes, but
> still relying on them is an equal bug.
> 
> > But if there exist some custom hw devices out there that are in the
> > same predicament as my DUART28 adapter, but are different in that they
> > are classic old-fashioned RS-232 rather than integrated USB-serial,
> > with no place to assign a custom USB ID, *then* we need a non-USB-ID-
> > dependent solution such as Johan's sysfs attribute or O_DIRECT.
> 
> Any device with a classic old-fashioned RS-232 has probably already
> solved this in another way or is accepted as not working on Linux.
> 
> And then there is also the device tree (overlay?) through which a quirk
> like this can be communicated to the kernel driver. Not sure if this
> could help for a plug-and-play device like on USB.
> 
> > > So I want to get a bunch of people
> > > to agree that this is ok to do things this way now before taking this
> > > new user-visible api.
> 
> Personally, I would prefer the VID:PID to enforce the quirk and an
> O_DIRECT (or other) flag used on open() as general backup plan. To
> me a sysfs solution seems illogical.

The "problem" of a vid:pid is that for usb-serial devices, that only
describes the device that does the conversion itself, NOT the serial
device the converter is plugged into that cares about these types of
line-wiggling.

Just like you would not want to classify all devices that met the PCI
serial class signature for this type of thing either, there is nothing
special about USB here other than it happens to be a common transport
for these signals these days.

thanks,

greg k-h
Maarten Brock Dec. 10, 2020, 12:05 p.m. UTC | #9
On 2020-12-10 11:50, Greg Kroah-Hartman wrote:
> On Thu, Dec 10, 2020 at 11:41:24AM +0100, Maarten Brock wrote:
>> Hello Mychaela,
>> 
>> On 2020-12-09 23:49, Mychaela Falconia wrote:
>> > Greg K-H wrote:
>> >
>> > > I think we need more review for the rest of the series.  This does
>> > > change the way serial ports work in a non-traditional way (i.e. using
>> > > sysfs instead of terminal settings).
>> >
>> > But the problem is that the current status quo is fundamentally broken
>> > for those hardware devices in which DTR and/or RTS have been repurposed
>> > for something other than modem and flow control.  Right now whenever a
>> > "cold" (never previously opened) serial port is opened for the first
>> > time, that open action immediately and unstoppably asserts both DTR
>> > and RTS hardware outputs, without giving userspace any opportunity to
>> > say "no, please don't do it".  Yes, this behaviour is codified in a
>> > bunch of standards that ultimately trace back to 1970s Original UNIX,
>> > but just because it is a standard does not make it right - this
>> > Unix/POSIX/Linux "standard" serial port behaviour is a bug, not a
>> > feature.
>> 
>> I agree. And an application not configuring the required handshakes, 
>> but
>> still relying on them is an equal bug.
>> 
>> > But if there exist some custom hw devices out there that are in the
>> > same predicament as my DUART28 adapter, but are different in that they
>> > are classic old-fashioned RS-232 rather than integrated USB-serial,
>> > with no place to assign a custom USB ID, *then* we need a non-USB-ID-
>> > dependent solution such as Johan's sysfs attribute or O_DIRECT.
>> 
>> Any device with a classic old-fashioned RS-232 has probably already
>> solved this in another way or is accepted as not working on Linux.
>> 
>> And then there is also the device tree (overlay?) through which a 
>> quirk
>> like this can be communicated to the kernel driver. Not sure if this
>> could help for a plug-and-play device like on USB.
>> 
>> > > So I want to get a bunch of people
>> > > to agree that this is ok to do things this way now before taking this
>> > > new user-visible api.
>> 
>> Personally, I would prefer the VID:PID to enforce the quirk and an
>> O_DIRECT (or other) flag used on open() as general backup plan. To
>> me a sysfs solution seems illogical.
> 
> The "problem" of a vid:pid is that for usb-serial devices, that only
> describes the device that does the conversion itself, NOT the serial
> device the converter is plugged into that cares about these types of
> line-wiggling.
> 
> Just like you would not want to classify all devices that met the PCI
> serial class signature for this type of thing either, there is nothing
> special about USB here other than it happens to be a common transport
> for these signals these days.
> 
> thanks,
> 
> greg k-h

This is true for a generic USB-UART board or cable, but not for a
dedicated PCB where both the USB-UART chip and the special connection
are implemented and which has a dedicated VID:PID different from any
generic one. In this case the VID:PID describes the whole board.

If the line-wiggling requirement is created behind some sort of
connector (real RS-232 DB9/DB25 or CMOS pin header or whatever)
then the problem is the same as for an 8250 on any other bus. For
this situation I would prefer the O_DIRECT flag on open().

Maarten
Greg Kroah-Hartman Dec. 10, 2020, 1:01 p.m. UTC | #10
On Thu, Dec 10, 2020 at 01:05:15PM +0100, Maarten Brock wrote:
> On 2020-12-10 11:50, Greg Kroah-Hartman wrote:
> > On Thu, Dec 10, 2020 at 11:41:24AM +0100, Maarten Brock wrote:
> > > Hello Mychaela,
> > > 
> > > On 2020-12-09 23:49, Mychaela Falconia wrote:
> > > > Greg K-H wrote:
> > > >
> > > > > I think we need more review for the rest of the series.  This does
> > > > > change the way serial ports work in a non-traditional way (i.e. using
> > > > > sysfs instead of terminal settings).
> > > >
> > > > But the problem is that the current status quo is fundamentally broken
> > > > for those hardware devices in which DTR and/or RTS have been repurposed
> > > > for something other than modem and flow control.  Right now whenever a
> > > > "cold" (never previously opened) serial port is opened for the first
> > > > time, that open action immediately and unstoppably asserts both DTR
> > > > and RTS hardware outputs, without giving userspace any opportunity to
> > > > say "no, please don't do it".  Yes, this behaviour is codified in a
> > > > bunch of standards that ultimately trace back to 1970s Original UNIX,
> > > > but just because it is a standard does not make it right - this
> > > > Unix/POSIX/Linux "standard" serial port behaviour is a bug, not a
> > > > feature.
> > > 
> > > I agree. And an application not configuring the required handshakes,
> > > but
> > > still relying on them is an equal bug.
> > > 
> > > > But if there exist some custom hw devices out there that are in the
> > > > same predicament as my DUART28 adapter, but are different in that they
> > > > are classic old-fashioned RS-232 rather than integrated USB-serial,
> > > > with no place to assign a custom USB ID, *then* we need a non-USB-ID-
> > > > dependent solution such as Johan's sysfs attribute or O_DIRECT.
> > > 
> > > Any device with a classic old-fashioned RS-232 has probably already
> > > solved this in another way or is accepted as not working on Linux.
> > > 
> > > And then there is also the device tree (overlay?) through which a
> > > quirk
> > > like this can be communicated to the kernel driver. Not sure if this
> > > could help for a plug-and-play device like on USB.
> > > 
> > > > > So I want to get a bunch of people
> > > > > to agree that this is ok to do things this way now before taking this
> > > > > new user-visible api.
> > > 
> > > Personally, I would prefer the VID:PID to enforce the quirk and an
> > > O_DIRECT (or other) flag used on open() as general backup plan. To
> > > me a sysfs solution seems illogical.
> > 
> > The "problem" of a vid:pid is that for usb-serial devices, that only
> > describes the device that does the conversion itself, NOT the serial
> > device the converter is plugged into that cares about these types of
> > line-wiggling.
> > 
> > Just like you would not want to classify all devices that met the PCI
> > serial class signature for this type of thing either, there is nothing
> > special about USB here other than it happens to be a common transport
> > for these signals these days.
> > 
> > thanks,
> > 
> > greg k-h
> 
> This is true for a generic USB-UART board or cable, but not for a
> dedicated PCB where both the USB-UART chip and the special connection
> are implemented and which has a dedicated VID:PID different from any
> generic one. In this case the VID:PID describes the whole board.

Companies/devices lie about vid:pid all the time, wait until your
specific vid:pid is repurposed for some other device and then what
happens?  :)

> If the line-wiggling requirement is created behind some sort of
> connector (real RS-232 DB9/DB25 or CMOS pin header or whatever)
> then the problem is the same as for an 8250 on any other bus. For
> this situation I would prefer the O_DIRECT flag on open().

O_DIRECT is an interesting hack, has anyone seen if it violates the
posix rules for us to use it on a character device like this?

thanks,

greg k-h
Mychaela Falconia Dec. 10, 2020, 6:59 p.m. UTC | #11
Greg K-H wrote:

> Thanks for the long response, but I think you have to realize that

> creating a new api for something that has been "how things work" since

> the 1970's should not be taken lightly.  No matter if it was a bug or

> not, changing user-visable behavior is not a trivial thing.


But I am NOT asking to change any user-visible behaviour on any
existing hardware made by anyone other than me, instead I am merely
asking to mainline a simple purely additive patch that adds support
for my custom FTDI device with a custom USB ID, to spare my users the
pain of applying that same patch locally on their own systems.

As a secondary objective, I am also contributing my thoughts toward a
generic solution for potential other affected devices that don't have
the luxury of a custom PCB with a fully embedded USB-serial chip - but
that one is more theoretical at the moment.

> What we

> come up with here has to stand the test of time of being able to be

> supported properly for the next 40+ years.


And how do we achieve this goal if the total number of people in the
world who care in any way at all can be counted on one hand?

Maarten Brock wrote:

> I agree. And an application not configuring the required handshakes, but

> still relying on them is an equal bug.


This comment can be interpreted in at least two different ways.  Are
you referring to:

1) Mainstream existing applications that expect DTR and/or RTS to be
asserted on open without doing any explicit TIOCMBIS,

or

2) My fc-host-tools programs (fc-loadtool, fc-xram, rvinterf etc) that
operate on the second UART of my DUART28C adapter, expect to NOT have
auto-assertion of DTR/RTS on open, but rely on my custom USB ID and
the ftdi_sio driver patch that goes with it to suppress this auto-
assertion, without doing any explicit TIOCMBIC.

If you are referring to 1, it is difficult to fault the authors of
those applications, as they had every right to depend on the behaviour
that had been codified in numerous official standards like POSIX.  Or
if you are referring to 2, what other choice do I have?  With existing
unpatched Linux kernels of every currently existing version, it is
*impossible* to prevent DTR & RTS auto-assertion immediately on open
of a tty device, thus applying a patch to the kernel (or at least to
the ftdi_sio driver in my case) is the *only* way to make my hardware
work with Linux.  Doing a TIOCMBIC after open won't help, as it will
be too late if the kernel already asserted DTR & RTS and thus caused
an unwanted deep reset.

> Any device with a classic old-fashioned RS-232 has probably already

> solved this in another way or is accepted as not working on Linux.


If someone built such a device for their own personal enjoyment rather
than for commercial sale, and needed to get it working with Linux, I
suspect that person most likely applied a local patch to the kernel
on their own system, likely implementing something similar to what is
being discussed in this thread.

> And then there is also the device tree (overlay?) through which a quirk

> like this can be communicated to the kernel driver.


This method should indeed work for embedded boards that feature some
kind of SoC where one or more native UARTs on that SoC need the quirk
applied to them.  But I don't have any such use cases, so let's defer
this approach until and unless there is at least one person in the
world who does.

> Not sure if this

> could help for a plug-and-play device like on USB.


I don't see how this method can be useful at all for devices that
connect to a standard (non-embedded) PC or laptop, whether USB or
RS-232.

> Personally, I would prefer the VID:PID to enforce the quirk and an

> O_DIRECT (or other) flag used on open() as general backup plan. To

> me a sysfs solution seems illogical.


A sysfs solution could work as a sort of poor man's substitute for a
VID:PID-driven quirk: instead of a driver quirk in the kernel, there
is a udev rule that detects a particular USB-serial device (perhaps
based on textual manuf/product strings as opposed to VID:PID) and sets
the needed sysfs flag.  But then if we are talking about a special
USB device as opposed to generic serial as in RS-232 etc, then I argue
for a driver quirk: if the device has a custom VID:PID, a patch to the
driver is needed in any case just to recognize that custom ID, so why
not support the custom hw device properly by setting the quirk bit
right there and then?  Seen in this light, the sysfs approach indeed
makes little sense.

OTOH, if we are talking about RS-232 or similarly interfaced devices
which the user plugs into any random serial port (PC native, or a
random off-the-shelf USB-serial cable), then there is really nothing
that a udev rule can key onto, it is just a user plugging in some
serial device and then running custom userspace apps on it.  In this
case asking the user to 'echo' something from the shell into /sys/blah
prior to running her userspace app seems illogical indeed, and asking
userspace app programmers to implement an equivalent sysfs write
internally is equally awkward.  For this non-custom-USB-ID scenario I
thus agree that the O_DIRECT approach would be better - in this case
the userspace app programmer simply needs to add this one flag to
their open call, a trivial one line change.

> > [Greg's objection to USB VID:PID quirks]

>

> This is true for a generic USB-UART board or cable, but not for a

> dedicated PCB where both the USB-UART chip and the special connection

> are implemented and which has a dedicated VID:PID different from any

> generic one. In this case the VID:PID describes the whole board.


Maarten's understanding is fully correct here: my DUART28C adapter is
a custom PCB that contains both the FT2232D USB-serial chip *and* the
special wiring whereby Channel B RTS & DTR outputs are repurposed into
CTL1 & CTL2 non-serial-related open drain control outputs, and my
custom VID:PID of 0x0403:0x7152 uniquely identifies this complete
custom-functionality PCB, not just the FT2232D part of it.

> If the line-wiggling requirement is created behind some sort of

> connector (real RS-232 DB9/DB25 or CMOS pin header or whatever)

> then the problem is the same as for an 8250 on any other bus. For

> this situation I would prefer the O_DIRECT flag on open().


I agree.

Greg K-H wrote:

> Companies/devices lie about vid:pid all the time,


Wait, are you accusing *me* of lying?  PID range 0x7150 through 0x7157
out of FTDI's VID 0x0403 has been officially allocated by FTDI (please
feel free to confirm with FTDI, no need to take my word blindly) to
Falconia Partners LLC, which is *my* company - Falconia is my last
name.  Therefore, any accusation of lying in connection with any
VID:PID in this range, including DUART28C VID:PID of 0x0403:0x7152, is
an accusation of lying against me personally, which I take very
seriously.

> wait until your

> specific vid:pid is repurposed for some other device and then what

> happens?  :)


Let's break it down:

1) I do have an idea for a future hw product that could indeed reuse
DUART28C VID:PID - however, the reason for reusing the same USB ID is
that the potential product in question would have exactly the same
wiring inside and require the exact same handling from Linux, i.e.,
suppress automatic DTR/RTS assertion on Channel B, but retain standard
behaviour on Channel A.

2) If I were to reuse the same USB ID for a different hw product that
should NOT receive the exact same quirk treatment, it would be 100% my
fault and I would have no right to run to OS maintainers complaining,
or even to sell such product, I would argue.

3) If some other party illegally squats on a PID out of FTDI's VID
range which FTDI officially allocated to me, and then comes to Linux
maintainers with a complaint however many years from now, whoever will
be the maintainer at that time will be able to check with FTDI, get an
official answer as to whom that PID rightfully belongs to, and that
will be the resolution.

> O_DIRECT is an interesting hack, has anyone seen if it violates the

> posix rules for us to use it on a character device like this?


According to open(2) Linux man page, O_DIRECT does not come from POSIX
at all, instead it is specific to Linux, FreeBSD and SGI IRIX.  Thus
it seems like there aren't any POSIX rules to be violated here.

If we go with O_DIRECT, what semantics are we going to implement?
There are 3 possibilities that come to mind most readily:

1) O_DIRECT applies only to the open call in which this flag is set,
and suppresses DTR/RTS assertion on that open.  If someone needs to do
multiple opens with DTR/RTS suppression being required every time,
then they need to include O_DIRECT every time.

2) O_DIRECT applies not only immediately, but also sets a latched flag
whereby all subsequent opens continue to suppress auto-assertion
without requiring O_DIRECT every time.  This approach by itself runs
counter to the generic Unix way of doing things, but it may be OK if
there is also some ioctl to explicitly set or clear the latched flag.

3) O_DIRECT applies only to the open call in which it is set, no
built-in latching, but there is also some ioctl to control a flag
enabling or disabling DTR/RTS auto-assertion on subsequent opens.

My vote would be to implement 1 first, for reasons of utmost
simplicity, both interface and implementation, and then possibly
implement 3 at some later time if and when someone actually needs that
extra bit of complexity.

A strong argument can be made that if someone repurposed DTR and/or
RTS signals on a serial port for some very different non-standard
purpose, then they will almost certainly be writing their own custom
userspace sw to talk to that serial port, as opposed to using standard
existing userspace sw, hence asking them to include a non-standard
flag on every open of their serial port shouldn't be too burdensome.

M~
Jiri Slaby Dec. 11, 2020, 8:46 a.m. UTC | #12
On 10. 12. 20, 19:59, Mychaela Falconia wrote:
>> O_DIRECT is an interesting hack, has anyone seen if it violates the

>> posix rules for us to use it on a character device like this?

> 

> According to open(2) Linux man page, O_DIRECT does not come from POSIX

> at all, instead it is specific to Linux, FreeBSD and SGI IRIX.  Thus

> it seems like there aren't any POSIX rules to be violated here.

> 

> If we go with O_DIRECT, what semantics are we going to implement?

> There are 3 possibilities that come to mind most readily:

> 

> 1) O_DIRECT applies only to the open call in which this flag is set,

> and suppresses DTR/RTS assertion on that open.  If someone needs to do

> multiple opens with DTR/RTS suppression being required every time,

> then they need to include O_DIRECT every time.

> 

> 2) O_DIRECT applies not only immediately, but also sets a latched flag

> whereby all subsequent opens continue to suppress auto-assertion

> without requiring O_DIRECT every time.  This approach by itself runs

> counter to the generic Unix way of doing things, but it may be OK if

> there is also some ioctl to explicitly set or clear the latched flag.

> 

> 3) O_DIRECT applies only to the open call in which it is set, no

> built-in latching, but there is also some ioctl to control a flag

> enabling or disabling DTR/RTS auto-assertion on subsequent opens.


3) -- to allow standard tools to work on the device after the quirk is 
set up once.

thanks,
-- 
js
Johan Hovold Dec. 11, 2020, 10:41 a.m. UTC | #13
On Thu, Dec 10, 2020 at 11:41:24AM +0100, Maarten Brock wrote:
> Hello Mychaela,

> 

> On 2020-12-09 23:49, Mychaela Falconia wrote:


> > But the problem is that the current status quo is fundamentally broken

> > for those hardware devices in which DTR and/or RTS have been repurposed

> > for something other than modem and flow control.  Right now whenever a

> > "cold" (never previously opened) serial port is opened for the first

> > time, that open action immediately and unstoppably asserts both DTR

> > and RTS hardware outputs, without giving userspace any opportunity to

> > say "no, please don't do it".  Yes, this behaviour is codified in a

> > bunch of standards that ultimately trace back to 1970s Original UNIX,

> > but just because it is a standard does not make it right - this

> > Unix/POSIX/Linux "standard" serial port behaviour is a bug, not a

> > feature.

> 

> I agree. And an application not configuring the required handshakes, but

> still relying on them is an equal bug.


Ehh, don't be silly. Relying on having Linux assert DTR/RTS on open is
in no way a bug; it's the standard behaviour that we must continue to
support unless explicitly overridden. Period.

> > But if there exist some custom hw devices out there that are in the

> > same predicament as my DUART28 adapter, but are different in that they

> > are classic old-fashioned RS-232 rather than integrated USB-serial,

> > with no place to assign a custom USB ID, *then* we need a non-USB-ID-

> > dependent solution such as Johan's sysfs attribute or O_DIRECT.

> 

> Any device with a classic old-fashioned RS-232 has probably already

> solved this in another way or is accepted as not working on Linux.

> 

> And then there is also the device tree (overlay?) through which a quirk

> like this can be communicated to the kernel driver. Not sure if this

> could help for a plug-and-play device like on USB.


Not every system use devicetree and no, it doesn't help with
hotpluggable buses either.

Johan
Johan Hovold Dec. 11, 2020, 10:46 a.m. UTC | #14
On Thu, Dec 10, 2020 at 02:01:30PM +0100, Greg Kroah-Hartman wrote:
> O_DIRECT is an interesting hack, has anyone seen if it violates the
> posix rules for us to use it on a character device like this?

Jiri only mentioned O_DIRECT as an example of a flag which we might be
able to repurpose/abuse for this. O_DIRECT is linux-specific, not in
POSIX, so we'd still end up with a Linux-specific interface if we were
to take this route.

Johan
Maarten Brock Dec. 11, 2020, 3:51 p.m. UTC | #15
On 2020-12-10 19:59, Mychaela Falconia wrote:
> Maarten Brock wrote:

> 

>> I agree. And an application not configuring the required handshakes, 

>> but

>> still relying on them is an equal bug.

> 

> This comment can be interpreted in at least two different ways.  Are

> you referring to:

> 

> 1) Mainstream existing applications that expect DTR and/or RTS to be

> asserted on open without doing any explicit TIOCMBIS,

> 

> or

> 

> 2) My fc-host-tools programs (fc-loadtool, fc-xram, rvinterf etc) that

> operate on the second UART of my DUART28C adapter, expect to NOT have

> auto-assertion of DTR/RTS on open, but rely on my custom USB ID and

> the ftdi_sio driver patch that goes with it to suppress this auto-

> assertion, without doing any explicit TIOCMBIC.

> 

> If you are referring to 1, it is difficult to fault the authors of

> those applications, as they had every right to depend on the behaviour

> that had been codified in numerous official standards like POSIX.  Or

> if you are referring to 2, what other choice do I have?  With existing

> unpatched Linux kernels of every currently existing version, it is

> *impossible* to prevent DTR & RTS auto-assertion immediately on open

> of a tty device, thus applying a patch to the kernel (or at least to

> the ftdi_sio driver in my case) is the *only* way to make my hardware

> work with Linux.  Doing a TIOCMBIC after open won't help, as it will

> be too late if the kernel already asserted DTR & RTS and thus caused

> an unwanted deep reset.


I was referring to 1) And I still think that an application that 
*relies*
on handshakes working should configure the handshakes, even if POSIX
promises they should be set up.

>> Any device with a classic old-fashioned RS-232 has probably already

>> solved this in another way or is accepted as not working on Linux.

> 

> If someone built such a device for their own personal enjoyment rather

> than for commercial sale, and needed to get it working with Linux, I

> suspect that person most likely applied a local patch to the kernel

> on their own system, likely implementing something similar to what is

> being discussed in this thread.


A person might have done that, a company will probably just not support
Linux or do a redesign with a different solution.

>> Personally, I would prefer the VID:PID to enforce the quirk and an

>> O_DIRECT (or other) flag used on open() as general backup plan. To

>> me a sysfs solution seems illogical.

> 

> A sysfs solution could work as a sort of poor man's substitute for a

> VID:PID-driven quirk: instead of a driver quirk in the kernel, there

> is a udev rule that detects a particular USB-serial device (perhaps

> based on textual manuf/product strings as opposed to VID:PID) and sets

> the needed sysfs flag.  But then if we are talking about a special

> USB device as opposed to generic serial as in RS-232 etc, then I argue

> for a driver quirk: if the device has a custom VID:PID, a patch to the

> driver is needed in any case just to recognize that custom ID, so why

> not support the custom hw device properly by setting the quirk bit

> right there and then?  Seen in this light, the sysfs approach indeed

> makes little sense.

> 

> OTOH, if we are talking about RS-232 or similarly interfaced devices

> which the user plugs into any random serial port (PC native, or a

> random off-the-shelf USB-serial cable), then there is really nothing

> that a udev rule can key onto, it is just a user plugging in some

> serial device and then running custom userspace apps on it.  In this

> case asking the user to 'echo' something from the shell into /sys/blah

> prior to running her userspace app seems illogical indeed, and asking

> userspace app programmers to implement an equivalent sysfs write

> internally is equally awkward.  For this non-custom-USB-ID scenario I

> thus agree that the O_DIRECT approach would be better - in this case

> the userspace app programmer simply needs to add this one flag to

> their open call, a trivial one line change.


Or use your option 3) mentioned below: open with O_DIRECT, use ioctl to
set the sticky flag and close before starting the application.

>> O_DIRECT is an interesting hack, has anyone seen if it violates the

>> posix rules for us to use it on a character device like this?

> 

> According to open(2) Linux man page, O_DIRECT does not come from POSIX

> at all, instead it is specific to Linux, FreeBSD and SGI IRIX.  Thus

> it seems like there aren't any POSIX rules to be violated here.

> 

> If we go with O_DIRECT, what semantics are we going to implement?

> There are 3 possibilities that come to mind most readily:

> 

> 1) O_DIRECT applies only to the open call in which this flag is set,

> and suppresses DTR/RTS assertion on that open.  If someone needs to do

> multiple opens with DTR/RTS suppression being required every time,

> then they need to include O_DIRECT every time.

> 

> 2) O_DIRECT applies not only immediately, but also sets a latched flag

> whereby all subsequent opens continue to suppress auto-assertion

> without requiring O_DIRECT every time.  This approach by itself runs

> counter to the generic Unix way of doing things, but it may be OK if

> there is also some ioctl to explicitly set or clear the latched flag.

> 

> 3) O_DIRECT applies only to the open call in which it is set, no

> built-in latching, but there is also some ioctl to control a flag

> enabling or disabling DTR/RTS auto-assertion on subsequent opens.

> 

> My vote would be to implement 1 first, for reasons of utmost

> simplicity, both interface and implementation, and then possibly

> implement 3 at some later time if and when someone actually needs that

> extra bit of complexity.

> 

> A strong argument can be made that if someone repurposed DTR and/or

> RTS signals on a serial port for some very different non-standard

> purpose, then they will almost certainly be writing their own custom

> userspace sw to talk to that serial port, as opposed to using standard

> existing userspace sw, hence asking them to include a non-standard

> flag on every open of their serial port shouldn't be too burdensome.

> 

> M~


Option 3) looks best to me.

Maarten
Greg Kroah-Hartman Dec. 11, 2020, 4:08 p.m. UTC | #16
On Thu, Dec 10, 2020 at 10:59:41AM -0800, Mychaela Falconia wrote:
> Greg K-H wrote:
> 
> > Companies/devices lie about vid:pid all the time,
> 
> Wait, are you accusing *me* of lying?  PID range 0x7150 through 0x7157
> out of FTDI's VID 0x0403 has been officially allocated by FTDI (please
> feel free to confirm with FTDI, no need to take my word blindly) to
> Falconia Partners LLC, which is *my* company - Falconia is my last
> name.  Therefore, any accusation of lying in connection with any
> VID:PID in this range, including DUART28C VID:PID of 0x0403:0x7152, is
> an accusation of lying against me personally, which I take very
> seriously.

I am not saying you are using an "invalid id", or copying anything, see
below:

> > wait until your
> > specific vid:pid is repurposed for some other device and then what
> > happens?  :)
> 
> Let's break it down:
> 
> 1) I do have an idea for a future hw product that could indeed reuse
> DUART28C VID:PID - however, the reason for reusing the same USB ID is
> that the potential product in question would have exactly the same
> wiring inside and require the exact same handling from Linux, i.e.,
> suppress automatic DTR/RTS assertion on Channel B, but retain standard
> behaviour on Channel A.
> 
> 2) If I were to reuse the same USB ID for a different hw product that
> should NOT receive the exact same quirk treatment, it would be 100% my
> fault and I would have no right to run to OS maintainers complaining,
> or even to sell such product, I would argue.
> 
> 3) If some other party illegally squats on a PID out of FTDI's VID
> range which FTDI officially allocated to me, and then comes to Linux
> maintainers with a complaint however many years from now, whoever will
> be the maintainer at that time will be able to check with FTDI, get an
> official answer as to whom that PID rightfully belongs to, and that
> will be the resolution.

We see devices that are "obviously" not the real vid/pid all the time in
the wild.  There's nothing "illegal" about another company using your
vid/pid, look at all of the ones out there already that use the FTDI
vendor id yet are "clones", same with pl2303 devices.

We also have fuzzing devices that spoof vid/pid pairs in order to test
kernel code, as well as being used as malicious devices to hack systems
or do other "fun" things.  Blindly trusting these numbers are something
we can no longer do.

The point being, while it is nice to trigger off of these values, beware
that it is not the only way that something like a userspace visable
change should trigger off of because this is something that a user wants
to have happen.  This also makes it easier as it "should" work for all
serial devices, and not be tied to specific hardware ids, requiring
kernel updates for new devices that want to do this.

hope this helps explain this a bit more.

> > O_DIRECT is an interesting hack, has anyone seen if it violates the
> > posix rules for us to use it on a character device like this?
> 
> According to open(2) Linux man page, O_DIRECT does not come from POSIX
> at all, instead it is specific to Linux, FreeBSD and SGI IRIX.  Thus
> it seems like there aren't any POSIX rules to be violated here.

Ah, for some reason I thought that newer POSIX releases finally defined
this, as vendors wanted that written down so they could "rely" on it.
Maybe not...

thanks,

greg k-h
Greg Kroah-Hartman Dec. 11, 2020, 4:08 p.m. UTC | #17
On Fri, Dec 11, 2020 at 09:46:54AM +0100, Jiri Slaby wrote:
> On 10. 12. 20, 19:59, Mychaela Falconia wrote:

> > > O_DIRECT is an interesting hack, has anyone seen if it violates the

> > > posix rules for us to use it on a character device like this?

> > 

> > According to open(2) Linux man page, O_DIRECT does not come from POSIX

> > at all, instead it is specific to Linux, FreeBSD and SGI IRIX.  Thus

> > it seems like there aren't any POSIX rules to be violated here.

> > 

> > If we go with O_DIRECT, what semantics are we going to implement?

> > There are 3 possibilities that come to mind most readily:

> > 

> > 1) O_DIRECT applies only to the open call in which this flag is set,

> > and suppresses DTR/RTS assertion on that open.  If someone needs to do

> > multiple opens with DTR/RTS suppression being required every time,

> > then they need to include O_DIRECT every time.

> > 

> > 2) O_DIRECT applies not only immediately, but also sets a latched flag

> > whereby all subsequent opens continue to suppress auto-assertion

> > without requiring O_DIRECT every time.  This approach by itself runs

> > counter to the generic Unix way of doing things, but it may be OK if

> > there is also some ioctl to explicitly set or clear the latched flag.

> > 

> > 3) O_DIRECT applies only to the open call in which it is set, no

> > built-in latching, but there is also some ioctl to control a flag

> > enabling or disabling DTR/RTS auto-assertion on subsequent opens.

> 

> 3) -- to allow standard tools to work on the device after the quirk is set

> up once.


I'm lost, what do you mean here?

thanks,

greg k-h
Johan Hovold Dec. 14, 2020, 2:21 p.m. UTC | #18
On Thu, Dec 10, 2020 at 10:59:41AM -0800, Mychaela Falconia wrote:

> Maarten Brock wrote:


> > Personally, I would prefer the VID:PID to enforce the quirk and an

> > O_DIRECT (or other) flag used on open() as general backup plan. To

> > me a sysfs solution seems illogical.

> 

> A sysfs solution could work as a sort of poor man's substitute for a

> VID:PID-driven quirk: instead of a driver quirk in the kernel, there

> is a udev rule that detects a particular USB-serial device (perhaps

> based on textual manuf/product strings as opposed to VID:PID) and sets

> the needed sysfs flag.  But then if we are talking about a special

> USB device as opposed to generic serial as in RS-232 etc, then I argue

> for a driver quirk: if the device has a custom VID:PID, a patch to the

> driver is needed in any case just to recognize that custom ID, so why

> not support the custom hw device properly by setting the quirk bit

> right there and then?  Seen in this light, the sysfs approach indeed

> makes little sense.

> 

> OTOH, if we are talking about RS-232 or similarly interfaced devices

> which the user plugs into any random serial port (PC native, or a

> random off-the-shelf USB-serial cable), then there is really nothing

> that a udev rule can key onto, it is just a user plugging in some

> serial device and then running custom userspace apps on it.  In this

> case asking the user to 'echo' something from the shell into /sys/blah

> prior to running her userspace app seems illogical indeed, and asking

> userspace app programmers to implement an equivalent sysfs write

> internally is equally awkward.  For this non-custom-USB-ID scenario I

> thus agree that the O_DIRECT approach would be better - in this case

> the userspace app programmer simply needs to add this one flag to

> their open call, a trivial one line change.


My reasons for proposing the NORDY sysfs interface *and* termios flag
are as follows:

 - The change should be minimal, but still support non-standard use of
   the modem control signals.

 - The interface should be compatible with using standard tools (e.g.
   echo and cat).

 - We control all other aspects of this (e.g. HUPCL) using termios and
   should continue to do so.

 - We cannot change the default behaviour so this NORDY flag needs to be
   disabled by default (i.e. hence the negation).

 - You cannot generally rely on the state of these lines before opening
   the port at least once, but for applications where this is possible
   and where even toggling them once is undesirable we *can* provide
   some out-of-band mechanism to change the default state of the NORDY
   flag (but we could also ignore this use case, keeping the status
   quo).

 - This could be an interface to control just the initial state of this
   flag after probe() or it can be used in parallel with the termios
   interface. The latter is what is implemented here.

 - As a bonus, using sysfs for this allows this feature to be used also
   before NORDY support has been added to the c libraries.

 - A sysfs interface also integrates well with udev and allows custom
   hardware that depend on this to be setup up once at boot regardless
   of whether an RS-232 or USB-serial interface is used.

Note that one of the BSDs recently added a termios flag with the same
semantics as the proposed NORDY which seems to suggest that this is
interface is indeed a natural one.

Side note: Also the Windows API has a setting for the state of these
lines *after* open (i.e. similar to a termios flag), and there are
reports of Windows users not expecting the lines to be raised on first
open (and behaviour changing between OS releases). For FTDI devices
there appears to be some driver-specific out-of-band mechanism in the
system properties for setting the default behaviour. Perhaps there's
also some general mechanism, I have no idea, I've never used this API.
But at least it doesn't seem to be as simple as Mychaela suggested
earlier, that Win32 got this right, for example, by never raising the
lines on open (by default).

> Greg K-H wrote:

> 

> > Companies/devices lie about vid:pid all the time,

> 

> Wait, are you accusing *me* of lying?  PID range 0x7150 through 0x7157

> out of FTDI's VID 0x0403 has been officially allocated by FTDI (please

> feel free to confirm with FTDI, no need to take my word blindly) to

> Falconia Partners LLC, which is *my* company - Falconia is my last

> name.  Therefore, any accusation of lying in connection with any

> VID:PID in this range, including DUART28C VID:PID of 0x0403:0x7152, is

> an accusation of lying against me personally, which I take very

> seriously.


Just chill, and then re-read what Greg wrote.

> > wait until your

> > specific vid:pid is repurposed for some other device and then what

> > happens?  :)


> > O_DIRECT is an interesting hack, has anyone seen if it violates the

> > posix rules for us to use it on a character device like this?

> 

> According to open(2) Linux man page, O_DIRECT does not come from POSIX

> at all, instead it is specific to Linux, FreeBSD and SGI IRIX.  Thus

> it seems like there aren't any POSIX rules to be violated here.


No, but we'd still be repurposing a flag which is used for something
entirely different.

> If we go with O_DIRECT, what semantics are we going to implement?

> There are 3 possibilities that come to mind most readily:

> 

> 1) O_DIRECT applies only to the open call in which this flag is set,

> and suppresses DTR/RTS assertion on that open.  If someone needs to do

> multiple opens with DTR/RTS suppression being required every time,

> then they need to include O_DIRECT every time.


This precludes using standard tools like echo/cat and terminal programs.

> 2) O_DIRECT applies not only immediately, but also sets a latched flag

> whereby all subsequent opens continue to suppress auto-assertion

> without requiring O_DIRECT every time.  This approach by itself runs

> counter to the generic Unix way of doing things, but it may be OK if

> there is also some ioctl to explicitly set or clear the latched flag.


Yeah, this is the worst option; you cannot even disable this feature
unless you provide an ioctl interface as well.
 
> 3) O_DIRECT applies only to the open call in which it is set, no

> built-in latching, but there is also some ioctl to control a flag

> enabling or disabling DTR/RTS auto-assertion on subsequent opens.


This would be my choice *if* we go this route.

> My vote would be to implement 1 first, for reasons of utmost

> simplicity, both interface and implementation, and then possibly

> implement 3 at some later time if and when someone actually needs that

> extra bit of complexity.


Option 1 is not generic enough, and essentially only allows for your
very specific use case (and not, for example, to suppress reset on open
when using various embedded boards with terminal programs/echo/cat).

> A strong argument can be made that if someone repurposed DTR and/or

> RTS signals on a serial port for some very different non-standard

> purpose, then they will almost certainly be writing their own custom

> userspace sw to talk to that serial port, as opposed to using standard

> existing userspace sw, hence asking them to include a non-standard

> flag on every open of their serial port shouldn't be too burdensome.


Not true, consider the various Arduino tinkerers, they may not even have
designed the reset circuit themselves.

Johan
Mychaela Falconia Dec. 18, 2020, 6:03 p.m. UTC | #19
Greg K-H wrote:

> We see devices that are "obviously" not the real vid/pid all the time in

> the wild.  There's nothing "illegal" about another company using your

> vid/pid, look at all of the ones out there already that use the FTDI

> vendor id yet are "clones", same with pl2303 devices.


But are those reusers of someone else's VID or PID coming to Linux
kernel maintainers with requests to modify ftdi_sio or pl2303 drivers
to work with their clones?  Do you ever see LKML posts along the lines
of "Hi, I am so and so from such and such company, we are not FTDI but
we reuse FTDI's VID and PIDs, but our clone chip does not match the
original and we need to modify the ftdi_sio driver to work with our
poor man's clone chip" - do reusers of someone else's VID or PID come
here with such requests?

Let's apply this logic to the very specific example of USB VID:PID
0x0403:0x7152.  Right now I am asking you, esteemed maintainers, to
accept my patch to ftdi_sio adding support for this new custom ID
along with the necessary quirk.  If my patch were accepted and then
someone else squats on this USB ID, how would it be a problem for
anyone other than that squatter?  Are you expecting that someone is
going to file a bug report saying "I am using 0x0403:0x7152 for my own
USB device, but there is a quirk for this ID in ftdi_sio, and I don't
want that quirk for MY device using that ID" - is this what you are
anticipating?  If that scenario were to occur, how could it possibly
be right to choose that hypothetical complainant's interest over mine
if one can trivially check with FTDI and confirm that 0x0403:0x7152 is
officially allocated to Falconia and not to the other contesting party?

> We also have fuzzing devices that spoof vid/pid pairs in order to test

> kernel code, as well as being used as malicious devices to hack systems

> or do other "fun" things.


And where is the harm?  If my patch adding ftdi_sio support for
0x0403:0x7152 with the special quirk I am asking for were to be
accepted, and then someone presents this USB ID to a target system as
a form of hack or fuzzing test, where is the harm?  If anything, such
action would be a good thing, as it would exercise the quirk function
and ensure that it doesn't crash the kernel or anything like that.
Yes, the quirk flag will be set on that ttyUSB device - but if they
are deliberately presenting a USB ID whose owner said "this ID means
that this quirk is needed", then how is it a bad thing to apply the
quirk when that ID is detected, fuzzing/hacking or not?

> Blindly trusting these numbers are something

> we can no longer do.


Please clarify what you mean by "blindly trusting" in this very
specific context.  Suppose some company makes an entirely new USB-
interfaced chip, they assign it a unique ID (obtained officially
through proper channels, not taking squatting on else's), and they
submit a brand new Linux driver for their brand new chip, a driver
that naturally binds to that chip's new unique ID - are you going to
reject that driver and keep the new chip forever unsupported by Linux
because you are concerned that someone else will reuse that same USB
ID for something completely different?  How is my case any different?
The only difference is that in my case the custom hw product is a
board and not a chip - but how are board-level designs any less worthy
of mainline Linux support than chip-level ones?

Johan Hovold wrote:

> My reasons for proposing the NORDY sysfs interface *and* termios flag

> are as follows:


The patch series presented so far provides the sysfs interface and the
ftdi_sio ID-code-specific quirk for DUART28C (which would make me
happy), but does not add any new termios flag.  Are you planning to
present a new patch series that also adds the new termios flag?

> You cannot generally rely on the state of these lines before opening

> the port at least once,


I disagree.  Manufacturers of USB-serial chips like FT232x/FT2232x
*guarantee* (see FTDI's AN 184) that their DTR/RTS CMOS outputs *will
not* drive low (they will either drive high or be tristated) until and
unless an explicit command comes in on the USB control pipe to change
those signals to the "on" state of CMOS low output.  Therefore,
hardware engineers have a right to demand from OS maintainers that
OSes (who are middlemen between applications and hw) provide a way to
NOT issue that particular USB control pipe command if it is not wanted
by the application.

> but for applications where this is possible

> and where even toggling them once is undesirable


s/undesirable/killer/, for the specific case of UART Channel B on
FreeCalypso DUART28C, the custom board with the custom USB ID at the
crux of my quest.

> we *can* provide

> some out-of-band mechanism to change the default state of the NORDY

> flag (but we could also ignore this use case, keeping the status

> quo).


In the case of the parenthetical option, are you basically saying
"screw you" to me and my users, refusing to mainline our zero-effect-
on-others USB ID-driven quirk patch, the one without which the device
cannot work with Linux?

> This could be an interface to control just the initial state of this

> flag after probe() or it can be used in parallel with the termios

> interface. The latter is what is implemented here.


My concern with the termios flag idea is that in the absence of a
specific proposed patch to review, I don't have a clear view of
exactly where this new flag will be inserted, and I have a concern
about this new flag breaking things.

Right now my custom userspace programs initialize termios like this:

#include <asm/ioctls.h>
#include <asm/termbits.h>

...

	struct termios2 target_termios;

	target_termios.c_iflag = IGNBRK;
	target_termios.c_oflag = 0;
	target_termios.c_cflag = br->termios_code | CLOCAL|HUPCL|CREAD|CS8;
	target_termios.c_lflag = 0;
	target_termios.c_cc[VMIN] = 1;
	target_termios.c_cc[VTIME] = 0;
	target_termios.c_ispeed = br->nonstd_speed;
	target_termios.c_ospeed = br->nonstd_speed;
	if (ioctl(target_fd, TCSETSF2, &target_termios) < 0) {
		perror("TCSETSF2");
		exit(1);
	}

br->termios_code will be codes like B115200 for standard baud rates,
or BOTHER for GSM-specific baud rates like 812500, with the actual bps
number going into br->nonstd_speed.  You mention C libraries below,
and there is an important lesson to be learned here: as I understand
it, support for non-standard baud rates via BOTHER was added to Linux
as in kernel ages ago (at the urging of embedded systems folks like
me), but libc people (whichever libc it is, I don't care for their
politics) apparently never got the memo, and it appears to be
impossible to set non-standard serial baud rates using standard libc
headers and termios APIs.  It appears that the *only* way to set a
non-standard serial port baud rate from a lowly Linux userspace app is
to use <asm/xxx.h> kernel headers and raw ioctls - and as I quickly
found out, the necessary <asm/xxx.h> headers *conflict* with standard
libc ones, hence the necessary code needs to be moved out into its own
compilation unit that doesn't need much in the way of other libc
headers.

Back to your proposed new termios flag: are you thinking about adding
it to c_cflag, where CLOCAL and HUPCL currently live?  If so, it would
be pretty much impossible for userspace code like mine to not clear
this new flag.  You could argue that my approach of absolutely setting
all termios fields is wrong, that I should read the previous termios
state and make my desired changes, but considering that most termios
flags are just unwanted noise for raw byte I/O applications, approaches
like mine (init termios from scratch) are probably commonplace.  OK,
I could read previous termios just to read and preserve the NORDY flag
- but how would I do it when it will be years (if ever) before the new
flag definition appears in userspace-available headers?

So here is my concern: suppose that the "generic" solution you guys
end up implementing and merging will revolve around a new termios flag
added to c_cflag or some similar place, along with a sysfs interface,
and maybe even an ftdi_sio driver quirk to set this flag automatically
when DUART28C custom USB ID is detected.  The first open will be good
- but the termios flag will get reset, and breakage will occur with
subsequent runs of the same userspace programs messing up the hw.  I
suppose I could implement a workaround by adding code to my FC host
tools userspace sw suite to always do a sysfs write before opening the
serial port, but it certainly isn't anywhere near clean.

If you still insist on a termios flag rather than just sysfs, one
possible clean solution would be to have two separate flags in the
kernel's internal data structures: the new termios flag would end up
inside struct ktermios in the kernel, whereas the flag manipulated via
sysfs or set in my DUART28C USB ID quirk goes into struct tty_port
iflags like in Johan's current patch.  If either flag is set, suppress
automatic DTR & RTS assertion on open.  Alternatively, put the new
flag under getserial/setserial Linux-specific ioctls, rather than
termios - then it won't get inadvertently cleared so easily.

> As a bonus, using sysfs for this allows this feature to be used also

> before NORDY support has been added to the c libraries.


This part is absolutely crucial: the new kernel feature will be
essentially useless if the lack of support in userspace-available
headers makes it inaccessible.  For devices other than my specific
DUART28C which I still argue should be covered by an ftdi_sio driver
quirk, sysfs is the only viable option for immediate relief of users'
suffering.

Oh, and here is another reason why a USB-ID-driven ftdi_sio quirk
patch is pretty much an absolute requirement for me: even if some
generic solution that works for "any" serial port were to be implemented
in current bleeding edge mainline, it will be years before these new
kernels percolate to end users to a sufficient degree to be considered
ubiquitous.  Thus for the foreseeable future, I *must* support users
of old kernels, and because the necessary feature does not exist there,
directing users to apply a local patch is the only option.  And if I
am telling users to apply a local patch to their kernels, that patch
needs to be minimally-invasive-surgical, self-contained entirely within
the ftdi_sio driver and not touching anything outside of it, and as
easy as possible to apply to a very wide range of old kernel versions.

Whatever solution you guys are going to come up with in terms of new
termios flags and/or sysfs, it will be infeasible to package it in a
form of a retro-patch which any Joe End User can trivially apply to
his or her old system running whatever random old kernel version - but
my self-contained ftdi_sio patch adding support for my custom USB ID
with the needed quirk *is* trivially applicable by end users on just
about any old system.  Thus I am begging and pleading with you to
accept my DUART28C USB ID quirk *in addition* to whatever "generic" or
"works for anyone" solution you esteemed gentlemen come up with.

> Note that one of the BSDs recently added a termios flag with the same

> semantics as the proposed NORDY which seems to suggest that this is

> interface is indeed a natural one.


Would you happen to have a link or at least some specific search
terms?  I would like to know which BSD it is, how they named their new
termios flag which you say does the same thing, and where they inserted
it, to get an idea of what to expect if going this route.

> Side note: Also the Windows API has a setting for the state of these

> lines *after* open (i.e. similar to a termios flag),


I would not describe it as being similar to a termios flag, instead I
would describe it as similar to TIOCMBIS and TIOCMBIC.  Perhaps they
retain the state on close-then-reopen sequences - I don't know that
part.

> and there are

> reports of Windows users not expecting the lines to be raised on first

> open


I read reports from Python users who were migrating apps from Windows
to Linux, expecting easy portability because it's Python, and they got
bitten by automatic DTR & RTS assertion under Linux which didn't
happen for them under Windows.  I don't work with Python at all, so I
don't know much in the way of details, but supposedly Python maintainers
responded that it is a known Linux limitation, that they can control
the state of lines only after initial open, and can't avoid an initial
glitch - whereas under Windows they did avoid that initial glitch,
going through layers of Python, not even using Win32 API directly.

> (and behaviour changing between OS releases). For FTDI devices

> there appears to be some driver-specific out-of-band mechanism in the

> system properties for setting the default behaviour.


I didn't know this part.

Mychaela,
she/her/hers
Greg Kroah-Hartman Jan. 7, 2021, 3:25 p.m. UTC | #20
On Fri, Dec 18, 2020 at 10:03:59AM -0800, Mychaela Falconia wrote:
> Greg K-H wrote:
> 
> > We see devices that are "obviously" not the real vid/pid all the time in
> > the wild.  There's nothing "illegal" about another company using your
> > vid/pid, look at all of the ones out there already that use the FTDI
> > vendor id yet are "clones", same with pl2303 devices.
> 
> But are those reusers of someone else's VID or PID coming to Linux
> kernel maintainers with requests to modify ftdi_sio or pl2303 drivers
> to work with their clones?  Do you ever see LKML posts along the lines
> of "Hi, I am so and so from such and such company, we are not FTDI but
> we reuse FTDI's VID and PIDs, but our clone chip does not match the
> original and we need to modify the ftdi_sio driver to work with our
> poor man's clone chip" - do reusers of someone else's VID or PID come
> here with such requests?

Users of devices with cloned ids come to us asking why their devices do
not work on Linux and how to fix them.  Happens all the time, and as the
job of a kernel is to enable hardware, we work to make this happen.

The vendors who do this are no where to be found, of course, and telling
people that the device they just purchased is "counterfit" doesn't
really fall in our job description.  We have all sorts of work-arounds
in Linux drivers to support "fake" devices, our job is to make Linux
work for everyone.

Anyway, this is far off-topic for this thread, sorry, just letting you
know why trying to rely on vid/pid is not the "final solution" people
might think it is.

I wish someone else would have weighed in on the different api options
here, oh well, let me think about this over the next week or so...

thanks,

greg k-h
Mychaela Falconia Jan. 31, 2021, 12:18 a.m. UTC | #21
Greg K-H wrote:

> our job is to make Linux work for everyone.

But as your refusal to accept the purely additive (zero impact on
anything other than specific hw in question) patch adding support for
a new hardware device clearly indicates, your job is NOT to make Linux
work for everyone, but rather for a smaller subset of "everyone"
*other than* hardware designers who come to the maintainers in good
faith, asking to mainline support for new hardware they just made.

Johan Hovold wrote:

> I couldn't find any such guarantee about the state of these pins when in
> output mode in the document you refer to, but that's besides the point.

The FTDI document in question shows when the pins in question
transition between actively driving outputs and tristate, making it
clear to hw engineers that the Hi-Z state must be handled gracefully,
practically meaning that pull-up resistors to the I/O voltage rail
need to the added.  But as one can trivially confirm with an
oscilloscope, the power-up transition is from tristate to driving
HIGH, not low - the *only* time when the chip will drive low on these
outputs is when the USB host explicitly commands it to, either by
turning on DTR/RTS in UART mode or by switching to one of the non-UART
modes in which these pins acquire different functions.

> First, there are other serial devices than those from FTDI.

The same principle applies to the initial power-up state of every
other reasonable UART chip on the planet.  Take the very original 8250
with good old +5V supply voltage - as you first apply power to the
chip, the DTR and RTS outputs will be TTL high (corresponding to
RS-232 inactive), and they *will not* drive low until and unless you
write into the Modem Control Register to change their state.

> Second,
> these lines can be in any state when the OS boots (e.g. set by a
> previous user or OS).

If the user is specifically working with a special hardware device
that does not tolerate DTR/RTS being randomly jerked around, it is
that user's responsibility to ensure that there is NO "previous user
or OS" anywhere in the picture.  Right now the only active use case
(active use case one meaning one with at least one live human actively
campaigning for it in the present time) is my DUART28C, and in this
active use case the USB-serial interface and the DTR/RTS-sensitive
hardware behind this interface are inseparably integrated (a single
PCB) into one special USB device.  This special USB device does not
exist at all to the host computer until the user plugs it in with the
intent of operating on it - so there is *no* "previous user or OS" to
talk about.

> In fact, we already do provide such a way and perhaps that is what we
> should use here. We can simply have the ftdi_sio driver not bind to
> your control interface, which isn't a serial interface to begin with
> anyway.
>
> Then you're free to use it in whatever way you prefer using libusb.

Not acceptable: even though DTR and RTS lines from FT2232D Channel B
are repurposed for a non-serial function, the main TxD & RxD lines on
that FT2232D channel do form a standard serial interface, and it needs
to work as such.

Your proposal to use libusb for all serial communication on the
secondary channel is unacceptable because it would require writing two
versions of every userspace program that needs to talk to the second
UART of the Calypso GSM chip: one special libusb-based version for use
with DUART28C, and the other standard /dev/ttyXXX version for every
other possible serial port to which the second UART of a Calypso GSM
chip can be connected, of which there is a great multitude.

One part that might not be obvious and probably needs clarification is
that the second UART of the Calypso GSM chip (made by TI, not me) is a
data-leads-only (TxD & RxD only) interface without any modem control
or even hw flow control capability, hence whenever it is connected to
any standard UART, that standard UART's modem control and flow control
signals are left unconnected - thus DTR and RTS outputs go nowhere.
"Standard" userspace software that talks to this UART interface (if
one can use the word "standard" for software that is specifically
written to speak the proprietary serial protocol of one particular
chip vendor) thus does absolutely nothing regarding DTR and RTS - does
not touch them in any way, beyond the kernel's automatic assertion on
open and negation on close.

My only innovation in the DUART28C adapter is to take these two
otherwise unused outputs and repurpose them for a different useful
function.  My userspace programs take -Pdtr and -Prts options that
cause these outputs to be pulsed; in the absence of these options
(when the very same userspace program is used to talk to the very same
TI chip going through a different hw path than my DUART28C), nothing
special is done with DTR or RTS, and the kernel's automatic diddling
of these lines poses no problem when they are unused and unconnected,
as happens with every standard serial port connected to a data-leads-
only UART on target hw.

But in any case, I give up now.  You have made it quite obvious that
there is nothing I can say or do to convince you to accept my patch
adding support for the DUART28C device, thus every user of this
hardware will have to keep applying the needed patch locally on their
own system for the rest of eternity.

The present DUART28 hardware is not an end user product, instead it is
an accessory supplied along with my GSM development boards.  The whole
setup involves multiple boards interconnected with ribbon cables, and
requires an ESD workstation (grounded mat and wrist strap) to use
safely, without destroying the hardware with ESD.  In other words, not
something that can be sold or given out to consumer-level people.

However, I am considering making an integrated USB-plug-in GSM modem
product, a gadget enclosed in a clear plastic box (hence no ESD
control measures needed from the user) with just a USB male-A
connector sticking out.  Internally the device will consist of the
USB-side circuit from DUART28 (FT2232D-based) connected directly to my
Calypso modem for GSM - Calypso has no native USB, just two UARTs.
This proposed device will be 100% compatible with current DUART28C and
will thus share the same custom USB ID - but unlike my current
development boards where everything is mostly modular, the boot
control interface of DUART28C will be absolutely required to operate
the new fully integrated modem gadget.  Hence we are going to have an
*end user* (as opposed to deep developers only) product that will
absolutely require my ftdi_sio patch in order to operate.

My earlier references to how Windows handles DTR and RTS were examples
for comparison - I am not a Windows programmer (the closest I come is
porting some programs from Windows to Linux, including programs that
talk via serial ports), I don't have a Windows version of any of my
programs for working with TI Calypso GSM gear, thus if I do produce
the end user gadget I am thinking about, it will be a product
specifically intended for Linux users.  And because you are refusing
to mainline my patch adding support for the hw in question, every user
will be expected to patch their kernel locally, regardless of how new
or old their kernel happens to be.  (If you were more reasonable, then
only old kernel users would need to apply the patch locally, whereas
new kernels from some point onward would just work out of the box -
but it is not to be, given your stance.)

Thus I am going to produce an end user product (a USB-plug-in GSM
modem similar to ubiquitous "data sticks", but with the special
distinction of fully published source code for GSM radio firmware)
specifically for Linux users, accompanied by freely-licensed
published-source userspace application software, rather than for users
of proprietary OSes like Windows where proprietary drivers and
applications are the expected norm.  But because you are refusing to
accept the hardware support patch which I submitted in good faith and
spent enormous amounts of mental and emotional energy defending and
justifying, I will have no option but to include a printed leaflet
with every piece of hardware explaining the following to the end users:

1) How to apply the needed ftdi_sio driver patch locally;

2) Links to these mailing list threads, showing that we as the
hardware vendor have really tried to do everything reasonable to work
with the mainline;

3) A suggestion for end users that they should direct their anger and
frustration at the specific maintainers who are refusing to accept the
simple new hw support patch.

Who knows, if you suddenly get hundreds of angry emails from end users
(not from me!) who are subjected to the pain of applying a local patch
solely because of your obstinence, perhaps you might budge then.

> https://reviews.freebsd.org/D20031

I admit that I don't know much about FreeBSD, but the following parts
from the above page caught my attention:

: This patch allows "stty -f /dev/ttyu9.init -rtsdtr" to prevent
: RTS and DTR from being asserted on open(), allowing these devices
: to be used without problems.

I draw your attention to the /dev/ttyu9.init device name - what exactly
does the .init part signify?

: .Pp
: If
: .Dv CNO_RTSDTR
: is set then the RTS and DTR lines will not be asserted when the device
: is opened.
: As a result, this flag is only useful on initial-state devices.

Does anyone know what is this "initial-state device" they are talking
about?  Is it perhaps a special logical device that only manipulates
software data structures in the kernel but does not touch the actual
hardware, such that opening the .init device for the stty operation
will *not* in itself cause the hardware lines to be jerked?  If it is
so, then FreeBSD has a much more fundamental feature (these "init state"
devices) which Linux lacks, and in this case my hardware should indeed
work with FreeBSD.

*If* the "initial state" devices in FreeBSD really do have the special
quality of only manipulating software (kernel) state without touching
the actual hardware, *then* despite the superficial similarity in
appearance (both using a termios flag and stty), what FreeBSD has done
is fundamentally different from what Johan has been proposing with his
version of termios flag idea.

Johan's idea is that it is perfectly OK to jerk DTR and RTS control
lines once for the stty operation.  I vehemently disagree with this
idea.  In many devices one of the lines can be wired as a reset - and
let's say your setup is part of a production environment where resetting
a live running system is not acceptable.  There is also the oft-cited
example of radios where DTR and/or RTS turns on the transmitter - now
suppose that radio is operating on some tightly regulated frequency
related to mission-critical services, and spurious out-of-protocol
transmissions are a big no-no.  In short, Johan's idea of it being OK
to jerk the control signals "just once" is NOT OK with me and never
will be, and I will continue to fight against this idea till my last
breath.

Sincerely,
Mychaela,
she/her/hers
Greg Kroah-Hartman Jan. 31, 2021, 5:24 a.m. UTC | #22
On Sat, Jan 30, 2021 at 04:18:04PM -0800, Mychaela Falconia wrote:
> Greg K-H wrote:
> 
> > our job is to make Linux work for everyone.
> 
> But as your refusal to accept the purely additive (zero impact on
> anything other than specific hw in question) patch adding support for
> a new hardware device clearly indicates, your job is NOT to make Linux
> work for everyone, but rather for a smaller subset of "everyone"
> *other than* hardware designers who come to the maintainers in good
> faith, asking to mainline support for new hardware they just made.

Anything we take adds work to our overall effort to support that new
feature added.  And you are asking us to do that work for you, for free,
for forever.  Sorry, given that your attitude does not understand that
this is a community and we need to work together, I don't think it's
worth continuing here, sorry.

If you change your mind in the future, you know how to contact us.

greg k-h