[tty-next,v4,5/6] serial: 8250: Switch to nbcon console

Implement the necessary callbacks to switch the 8250 console driver
to perform as an nbcon console.

Add implementations for the nbcon console callbacks:

  ->write_atomic()
  ->write_thread()
  ->device_lock()
  ->device_unlock()

and add CON_NBCON to the initial @flags.

All register access in the callbacks are within unsafe sections.
The ->write_atomic and ->write_therad() callbacks allow safe
handover/takeover per byte and add a preceding newline if they
take over from another context mid-line.

For the ->write_atomic() callback, a new irq_work is used to defer
modem control since it may be called from a context that does not
allow waking up tasks.

Note: A new __serial8250_clear_IER() is introduced for direct
clearing of UART_IER. This will allow to restore the lockdep
check to serial8250_clear_IER() in a follow-up commit.

Signed-off-by: John Ogness <john.ogness@linutronix.de>
---
 drivers/tty/serial/8250/8250_core.c |  35 ++++++-
 drivers/tty/serial/8250/8250_port.c | 149 +++++++++++++++++++++-------
 include/linux/serial_8250.h         |   8 +-
 3 files changed, 150 insertions(+), 42 deletions(-)

On Fri, Dec 27, 2024 at 11:51:21PM +0106, John Ogness wrote:
> Implement the necessary callbacks to switch the 8250 console driver
> to perform as an nbcon console.
> 
> Add implementations for the nbcon console callbacks:
> 
>   ->write_atomic()
>   ->write_thread()
>   ->device_lock()
>   ->device_unlock()
> 
> and add CON_NBCON to the initial @flags.
> 
> All register access in the callbacks are within unsafe sections.
> The ->write_atomic and ->write_therad() callbacks allow safe

->write_atomic()

> handover/takeover per byte and add a preceding newline if they
> take over from another context mid-line.
> 
> For the ->write_atomic() callback, a new irq_work is used to defer
> modem control since it may be called from a context that does not
> allow waking up tasks.
> 
> Note: A new __serial8250_clear_IER() is introduced for direct
> clearing of UART_IER. This will allow to restore the lockdep
> check to serial8250_clear_IER() in a follow-up commit.

...

>  		if (toggle_ier) {
> +			/*
> +			 * Port locked to synchronize UART_IER access
> +			 * against the console

Missing period in multi-line comment.

> +			 */
> +			lockdep_assert_held_once(&p->port.lock);
> +
>  			p->ier |= UART_IER_RLSI | UART_IER_RDI;
>  			serial_port_out(&p->port, UART_IER, p->ier);
>  		}

...

> -static void fifo_wait_for_lsr(struct uart_8250_port *up, unsigned int count)
> +static int fifo_wait_for_lsr(struct uart_8250_port *up,
> +			      struct nbcon_write_context *wctxt,
> +			      unsigned int count)
>  {
>  	unsigned int i;
>  
>  	for (i = 0; i < count; i++) {
> +		if (!nbcon_can_proceed(wctxt))
> +			return -EPERM;
> +
>  		if (wait_for_lsr(up, UART_LSR_THRE))
> -			return;
> +			return 0;
>  	}
> +
> +	return -ETIMEDOUT;
>  }

...

>  static void serial8250_console_fifo_write(struct uart_8250_port *up,
> -					  const char *s, unsigned int count)
> +					  struct nbcon_write_context *wctxt)
>  {
> -	const char *end = s + count;
>  	unsigned int fifosize = up->tx_loadsz;
>  	struct uart_port *port = &up->port;
> +	const char *s = wctxt->outbuf;
> +	const char *end = s + wctxt->len;
>  	unsigned int tx_count = 0;
>  	bool cr_sent = false;
>  	unsigned int i;
>  
>  	while (s != end) {
>  		/* Allow timeout for each byte of a possibly full FIFO */
> -		fifo_wait_for_lsr(up, fifosize);
> +		if (fifo_wait_for_lsr(up, wctxt, fifosize) == -EPERM)
> +			return;

Perhaps it was already discussed and I forgot about it or hadn't read,
but I don't get how per-byte check for NBCON permission can help if there
is something already in the HW FIFO?

I mean in _this_ case wouldn't be enough to check FIFO to drain and only after
that check the permission?

>  		for (i = 0; i < fifosize && s != end; ++i) {
>  			if (*s == '\n' && !cr_sent) {

>  	 * Allow timeout for each byte written since the caller will only wait
>  	 * for UART_LSR_BOTH_EMPTY using the timeout of a single character
>  	 */
> -	fifo_wait_for_lsr(up, tx_count);
> +	fifo_wait_for_lsr(up, wctxt, tx_count);
> +}


...

> +	if (up->msr_saved_flags) {
> +		/*
> +		 * For atomic, it must be deferred to irq_work because this
> +		 * may be a context that does not permit waking up tasks.
> +		 */
> +		if (is_atomic)
> +			irq_work_queue(&up->modem_status_work);
> +		else
> +			serial8250_modem_status(up);

Hmm... Shouldn't this be part of _modem_status() for all drivers using this call?

> +	}

...

> +	bool			console_line_ended;	/* line fully output */

Sorry, I didn't get the comment. Do you meant "line is fully printed out
[to HW]"?

On 2024-12-29, Andy Shevchenko <andriy.shevchenko@linux.intel.com> wrote:
>> -static void fifo_wait_for_lsr(struct uart_8250_port *up, unsigned int count)
>> +static int fifo_wait_for_lsr(struct uart_8250_port *up,
>> +			      struct nbcon_write_context *wctxt,
>> +			      unsigned int count)
>>  {
>>  	unsigned int i;
>>  
>>  	for (i = 0; i < count; i++) {
>> +		if (!nbcon_can_proceed(wctxt))
>> +			return -EPERM;
>> +
>>  		if (wait_for_lsr(up, UART_LSR_THRE))
>> -			return;
>> +			return 0;
>>  	}
>> +
>> +	return -ETIMEDOUT;
>>  }
>
> ...
>
>>  static void serial8250_console_fifo_write(struct uart_8250_port *up,
>> -					  const char *s, unsigned int count)
>> +					  struct nbcon_write_context *wctxt)
>>  {
>> -	const char *end = s + count;
>>  	unsigned int fifosize = up->tx_loadsz;
>>  	struct uart_port *port = &up->port;
>> +	const char *s = wctxt->outbuf;
>> +	const char *end = s + wctxt->len;
>>  	unsigned int tx_count = 0;
>>  	bool cr_sent = false;
>>  	unsigned int i;
>>  
>>  	while (s != end) {
>>  		/* Allow timeout for each byte of a possibly full FIFO */
>> -		fifo_wait_for_lsr(up, fifosize);
>> +		if (fifo_wait_for_lsr(up, wctxt, fifosize) == -EPERM)
>> +			return;
>
> Perhaps it was already discussed and I forgot about it or hadn't read,
> but I don't get how per-byte check for NBCON permission can help if there
> is something already in the HW FIFO?
>
> I mean in _this_ case wouldn't be enough to check FIFO to drain and only after
> that check the permission?

If we did that and other CPUs had taken over printing, then this CPU
would possibly busy-wait the maximum timeout because the FIFO keeps
getting refilled by the other CPUs. If this CPU knows that it has lost
ownership then it can abort waiting and trying to write ASAP.

The CPU taking over will perform the necessary waiting for the FIFO to
drain what this CPU had wrote into the FIFO.

>
>>  		for (i = 0; i < fifosize && s != end; ++i) {

That being said, this loop is not checking for lost ownership between
bytes. I will add an nbcon_can_proceed() check here as well. If
ownership was lost, this CPU must not continue writing into the FIFO.

	for (i = 0; i < fifosize && s != end && nbcon_can_proceed(wctxt); ++i) {

>>  			if (*s == '\n' && !cr_sent) {
>
>>  	 * Allow timeout for each byte written since the caller will only wait
>>  	 * for UART_LSR_BOTH_EMPTY using the timeout of a single character
>>  	 */
>> -	fifo_wait_for_lsr(up, tx_count);
>> +	fifo_wait_for_lsr(up, wctxt, tx_count);
>> +}
>
>
> ...
>
>> +	if (up->msr_saved_flags) {
>> +		/*
>> +		 * For atomic, it must be deferred to irq_work because this
>> +		 * may be a context that does not permit waking up tasks.
>> +		 */
>> +		if (is_atomic)
>> +			irq_work_queue(&up->modem_status_work);
>> +		else
>> +			serial8250_modem_status(up);
>
> Hmm... Shouldn't this be part of _modem_status() for all drivers using this call?

I am not sure I follow what you are saying.

Only NBCON drivers (that have implemented the ->write_atomic() callback)
will print from any context. OTOH legacy console drivers do not print
directly from scheduling or NMI contexts and instead defer to a printk
irq_work.

As we convert more console drivers to NBCON, it might make sense to move
the irq_work into the generic struct uart_port and possibly consolidate
the _modem_status() implementations. They seem to be mostly copy/paste
code.

>> +	bool			console_line_ended;	/* line fully output */
>
> Sorry, I didn't get the comment. Do you meant "line is fully printed out
> [to HW]"?

'\n' was the most recent byte written to the TX register by the console
driver.

Tracking this is necessary during takeovers so that the CPU taking over
can politely add an extra newline for cleaner output.

The wording for this comment was suggested by Petr Mladek. Please
suggest an acceptable alternative if you require something more concise.

John

On Fri 2024-12-27 23:51:21, John Ogness wrote:
> Implement the necessary callbacks to switch the 8250 console driver
> to perform as an nbcon console.
> 
> Add implementations for the nbcon console callbacks:
> 
>   ->write_atomic()
>   ->write_thread()
>   ->device_lock()
>   ->device_unlock()
> 
> and add CON_NBCON to the initial @flags.
> 
> All register access in the callbacks are within unsafe sections.
> The ->write_atomic and ->write_therad() callbacks allow safe
> handover/takeover per byte and add a preceding newline if they
> take over from another context mid-line.
> 
> For the ->write_atomic() callback, a new irq_work is used to defer
> modem control since it may be called from a context that does not
> allow waking up tasks.
> 
> Note: A new __serial8250_clear_IER() is introduced for direct
> clearing of UART_IER. This will allow to restore the lockdep
> check to serial8250_clear_IER() in a follow-up commit.
> 
> --- a/drivers/tty/serial/8250/8250_port.c
> +++ b/drivers/tty/serial/8250/8250_port.c
> @@ -1406,9 +1416,6 @@ void serial8250_em485_stop_tx(struct uart_8250_port *p, bool toggle_ier)
>  {
>  	unsigned char mcr = serial8250_in_MCR(p);
>  
> -	/* Port locked to synchronize UART_IER access against the console. */
> -	lockdep_assert_held_once(&p->port.lock);

We should explain why it is OK to move the assert.

IMHO, most poeple would not understand the port lock is needed only
for UART_IER manipulation and not for UART_MCR manipulation.

We should probably explain that even the UART_MCR manipulation
is synchronized either by the port lock or by nbcon context
ownership. Where the nbcon context owner ship actually provides
synchronization against the port lock in all situations
except for the final unsafe flush in panic().

In the ideal world it would be nice to check the nbcon context
ownership. But it would require passing struct nbcon_write_context
which does not fit well in this low level API.

A comment might be enough.

Or do I miss something?

> -
>  	if (p->port.rs485.flags & SER_RS485_RTS_AFTER_SEND)
>  		mcr |= UART_MCR_RTS;
>  	else
> @@ -3339,14 +3356,21 @@ static void serial8250_console_restore(struct uart_8250_port *up)
>  	serial8250_out_MCR(up, up->mcr | UART_MCR_DTR | UART_MCR_RTS);
>  }
>  
> -static void fifo_wait_for_lsr(struct uart_8250_port *up, unsigned int count)
> +static int fifo_wait_for_lsr(struct uart_8250_port *up,
> +			      struct nbcon_write_context *wctxt,
> +			      unsigned int count)
>  {
>  	unsigned int i;
>  
>  	for (i = 0; i < count; i++) {

This would deserve a comment. The following comes to my mind:

		/*
		 * Pass the ownership as quickly as possible to a higher
		 * priority context. Otherwise, its attempt to take
		 * over the ownership might timeout. The new owner
		 * will wait for lsr before reusing the fifo.
		 */

> +		if (!nbcon_can_proceed(wctxt))
> +			return -EPERM;
> +
>  		if (wait_for_lsr(up, UART_LSR_THRE))
> -			return;
> +			return 0;
>  	}
> +
> +	return -ETIMEDOUT;
>  }
>  
>  /*
> @@ -3356,18 +3380,20 @@ static void fifo_wait_for_lsr(struct uart_8250_port *up, unsigned int count)
>   * to get empty.
>   */
>  static void serial8250_console_fifo_write(struct uart_8250_port *up,
> -					  const char *s, unsigned int count)
> +					  struct nbcon_write_context *wctxt)
>  {
> -	const char *end = s + count;
>  	unsigned int fifosize = up->tx_loadsz;
>  	struct uart_port *port = &up->port;
> +	const char *s = wctxt->outbuf;
> +	const char *end = s + wctxt->len;
>  	unsigned int tx_count = 0;
>  	bool cr_sent = false;
>  	unsigned int i;
>  
>  	while (s != end) {
>  		/* Allow timeout for each byte of a possibly full FIFO */
> -		fifo_wait_for_lsr(up, fifosize);
> +		if (fifo_wait_for_lsr(up, wctxt, fifosize) == -EPERM)
> +			return;
>  
>  		for (i = 0; i < fifosize && s != end; ++i) {
>  			if (*s == '\n' && !cr_sent) {

We might want to check nbcon_can_proceed() after each character.
I think that you have already suggested this in the reply to Andy.

But maybe, it is not that important because filling the FIFO buffer is
probably fast.

> --- a/include/linux/serial_8250.h
> +++ b/include/linux/serial_8250.h
> @@ -150,8 +150,12 @@ struct uart_8250_port {
>  #define LSR_SAVE_FLAGS UART_LSR_BRK_ERROR_BITS
>  	u16			lsr_saved_flags;
>  	u16			lsr_save_mask;
> +
> +	bool			console_line_ended;	/* line fully output */

I agree with Andy that the comment is confusing. I am sorry if it was
my proposal ;-)

I wonder if the following is better:

	bool			console_line_ended;	/* finished writing full line */

> +
>  #define MSR_SAVE_FLAGS UART_MSR_ANY_DELTA
>  	unsigned char		msr_saved_flags;
> +	struct irq_work		modem_status_work;
>  
>  	struct uart_8250_dma	*dma;
>  	const struct uart_8250_ops *ops;

Otherwise, it looks good. Well, I might want to double check it
on Monday morning.

Best Regards,
Petr