From patchwork Fri Feb 13 11:53:14 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Slaby X-Patchwork-Id: 44673 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-la0-f69.google.com (mail-la0-f69.google.com [209.85.215.69]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 9474C21527 for ; Fri, 13 Feb 2015 11:53:23 +0000 (UTC) Received: by lamq1 with SMTP id q1sf10352096lam.0 for ; Fri, 13 Feb 2015 03:53:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:sender:precedence:list-id:x-original-sender :x-original-authentication-results:mailing-list:list-post:list-help :list-archive:list-unsubscribe; bh=3DFumLcnIpwSxrY6RuYJEr6vFr9W45L5qVM11Ym2eWQ=; b=LYDFY/3CtSvPWUPOxFm2f24tPuUK4uhyS4VF42px0ShVIHshAd0X3zaqi50kj8qmOS v1vOekQn09utkM4uovFDUTLWkoUamZE2NWVL5FC7m/doSS3pBePMsWGv1bOvZX7D98b4 eoDTR1kuTlaTIg1FJEhG4BSUag8uW+AHHkaQZJ2rRBrh5DT1Dp6swISPS2roFKbMvZYt feaKUUFKydP7GvCpj3WxrDO1HKIz3UVo/8MsvWS1yMMoq3YMmVfNHbsw4pkJeyR0FfGi 1TDzj76wDG/yzKzrd3e4HAi+n79PSpPsshgs9j2G2TNHJLb453LRvbkEUkbwNIv2q65q MPOA== X-Gm-Message-State: ALoCoQlY6kaYr+sLCSUJNwI6QNOhT2kFh+3yYVIBLNZOQEVBnH9q9856use3L77q+ZKCxSzvYZVx X-Received: by 10.152.26.74 with SMTP id j10mr1292713lag.10.1423828402549; Fri, 13 Feb 2015 03:53:22 -0800 (PST) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.29.66 with SMTP id i2ls318459lah.23.gmail; Fri, 13 Feb 2015 03:53:22 -0800 (PST) X-Received: by 10.112.188.227 with SMTP id gd3mr7568166lbc.22.1423828402365; Fri, 13 Feb 2015 03:53:22 -0800 (PST) Received: from mail-lb0-f179.google.com (mail-lb0-f179.google.com. [209.85.217.179]) by mx.google.com with ESMTPS id eq1si1456575lac.11.2015.02.13.03.53.22 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 13 Feb 2015 03:53:22 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.179 as permitted sender) client-ip=209.85.217.179; Received: by mail-lb0-f179.google.com with SMTP id w7so15005739lbi.10 for ; Fri, 13 Feb 2015 03:53:22 -0800 (PST) X-Received: by 10.152.185.72 with SMTP id fa8mr7489060lac.73.1423828402275; Fri, 13 Feb 2015 03:53:22 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.35.133 with SMTP id h5csp1020077lbj; Fri, 13 Feb 2015 03:53:21 -0800 (PST) X-Received: by 10.70.135.165 with SMTP id pt5mr14215637pdb.101.1423828400561; Fri, 13 Feb 2015 03:53:20 -0800 (PST) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id lp9si2563502pab.184.2015.02.13.03.53.19; Fri, 13 Feb 2015 03:53:20 -0800 (PST) Received-SPF: none (google.com: stable-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752843AbbBMLxS (ORCPT + 1 other); Fri, 13 Feb 2015 06:53:18 -0500 Received: from ip4-83-240-67-251.cust.nbox.cz ([83.240.67.251]:35358 "EHLO ip4-83-240-18-248.cust.nbox.cz" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752585AbbBMLxR (ORCPT ); Fri, 13 Feb 2015 06:53:17 -0500 Received: from ku by ip4-83-240-18-248.cust.nbox.cz with local (Exim 4.85) (envelope-from ) id 1YMEo7-0001Pf-4r; Fri, 13 Feb 2015 12:53:15 +0100 From: Jiri Slaby To: stable@vger.kernel.org Cc: Alex Elder , Jiri Slaby Subject: [patch added to the 3.12 stable tree] rbd: drop an unsafe assertion Date: Fri, 13 Feb 2015 12:53:14 +0100 Message-Id: <1423828395-5396-1-git-send-email-jslaby@suse.cz> X-Mailer: git-send-email 2.2.2 Sender: stable-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: stable@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: patch@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.179 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , From: Alex Elder This patch has been added to the 3.12 stable tree. If you have any objections, please let us know. =============== commit 638c323c4d1f8eaf25224946e21ce8818f1bcee1 upstream. Olivier Bonvalet reported having repeated crashes due to a failed assertion he was hitting in rbd_img_obj_callback(): Assertion failure in rbd_img_obj_callback() at line 2165: rbd_assert(which >= img_request->next_completion); With a lot of help from Olivier with reproducing the problem we were able to determine the object and image requests had already been completed (and often freed) at the point the assertion failed. There was a great deal of discussion on the ceph-devel mailing list about this. The problem only arose when there were two (or more) object requests in an image request, and the problem was always seen when the second request was being completed. The problem is due to a race in the window between setting the "done" flag on an object request and checking the image request's next completion value. When the first object request completes, it checks to see if its successor request is marked "done", and if so, that request is also completed. In the process, the image request's next_completion value is updated to reflect that both the first and second requests are completed. By the time the second request is able to check the next_completion value, it has been set to a value *greater* than its own "which" value, which caused an assertion to fail. Fix this problem by skipping over any completion processing unless the completing object request is the next one expected. Test only for inequality (not >=), and eliminate the bad assertion. Tested-by: Olivier Bonvalet Signed-off-by: Alex Elder Reviewed-by: Sage Weil Reviewed-by: Ilya Dryomov Signed-off-by: Jiri Slaby --- drivers/block/rbd.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 5544f254175d..2eb4458f4ba8 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -2137,7 +2137,6 @@ static void rbd_img_obj_callback(struct rbd_obj_request *obj_request) rbd_assert(img_request->obj_request_count > 0); rbd_assert(which != BAD_WHICH); rbd_assert(which < img_request->obj_request_count); - rbd_assert(which >= img_request->next_completion); spin_lock_irq(&img_request->completion_lock); if (which != img_request->next_completion)