What: /sys/class/mic/
-Date: August 2013
-KernelVersion: 3.11
+Date: October 2013
+KernelVersion: 3.13
Contact: Sudeep Dutt <sudeep.dutt@intel.com>
Description:
The mic class directory belongs to Intel MIC devices and
Integrated Core (MIC) architecture that runs a Linux OS.
What: /sys/class/mic/mic(x)
-Date: August 2013
-KernelVersion: 3.11
+Date: October 2013
+KernelVersion: 3.13
Contact: Sudeep Dutt <sudeep.dutt@intel.com>
Description:
The directories /sys/class/mic/mic0, /sys/class/mic/mic1 etc.,
information specific to that MIC device.
What: /sys/class/mic/mic(x)/family
-Date: August 2013
-KernelVersion: 3.11
+Date: October 2013
+KernelVersion: 3.13
Contact: Sudeep Dutt <sudeep.dutt@intel.com>
Description:
Provides information about the Coprocessor family for an Intel
MIC device. For example - "x100"
What: /sys/class/mic/mic(x)/stepping
-Date: August 2013
-KernelVersion: 3.11
+Date: October 2013
+KernelVersion: 3.13
Contact: Sudeep Dutt <sudeep.dutt@intel.com>
Description:
Provides information about the silicon stepping for an Intel
MIC device. For example - "A0" or "B0"
What: /sys/class/mic/mic(x)/state
-Date: August 2013
-KernelVersion: 3.11
+Date: October 2013
+KernelVersion: 3.13
Contact: Sudeep Dutt <sudeep.dutt@intel.com>
Description:
When read, this entry provides the current state of an Intel
MIC device in the context of the card OS. Possible values that
will be read are:
- "offline" - The MIC device is ready to boot the card OS.
+ "offline" - The MIC device is ready to boot the card OS. On
+ reading this entry after an OSPM resume, a "boot" has to be
+ written to this entry if the card was previously shutdown
+ during OSPM suspend.
"online" - The MIC device has initiated booting a card OS.
"shutting_down" - The card OS is shutting down.
"reset_failed" - The MIC device has failed to reset.
+ "suspending" - The MIC device is currently being prepared for
+ suspend. On reading this entry, a "suspend" has to be written
+ to the state sysfs entry to ensure the card is shutdown during
+ OSPM suspend.
+ "suspended" - The MIC device has been suspended.
When written, this sysfs entry triggers different state change
operations depending upon the current state of the card OS.
sysfs entries.
"reset" - Initiates device reset.
"shutdown" - Initiates card OS shutdown.
+ "suspend" - Initiates card OS shutdown and also marks the card
+ as suspended.
What: /sys/class/mic/mic(x)/shutdown_status
-Date: August 2013
-KernelVersion: 3.11
+Date: October 2013
+KernelVersion: 3.13
Contact: Sudeep Dutt <sudeep.dutt@intel.com>
Description:
An Intel MIC device runs a Linux OS during its operation. This
"restart" - Shutdown because of a restart command.
What: /sys/class/mic/mic(x)/cmdline
-Date: August 2013
-KernelVersion: 3.11
+Date: October 2013
+KernelVersion: 3.13
Contact: Sudeep Dutt <sudeep.dutt@intel.com>
Description:
An Intel MIC device runs a Linux OS during its operation. Before
line back to this entry.
What: /sys/class/mic/mic(x)/firmware
-Date: August 2013
-KernelVersion: 3.11
+Date: October 2013
+KernelVersion: 3.13
Contact: Sudeep Dutt <sudeep.dutt@intel.com>
Description:
When read, this sysfs entry provides the path name under
firmware image location under /lib/firmware/.
What: /sys/class/mic/mic(x)/ramdisk
-Date: August 2013
-KernelVersion: 3.11
+Date: October 2013
+KernelVersion: 3.13
Contact: Sudeep Dutt <sudeep.dutt@intel.com>
Description:
When read, this sysfs entry provides the path name under
the ramdisk image location under /lib/firmware/.
What: /sys/class/mic/mic(x)/bootmode
-Date: August 2013
-KernelVersion: 3.11
+Date: October 2013
+KernelVersion: 3.13
Contact: Sudeep Dutt <sudeep.dutt@intel.com>
Description:
When read, this sysfs entry provides the current bootmode for
b) elf - Boot an elf image for flash updates.
What: /sys/class/mic/mic(x)/log_buf_addr
-Date: August 2013
-KernelVersion: 3.11
+Date: October 2013
+KernelVersion: 3.13
Contact: Sudeep Dutt <sudeep.dutt@intel.com>
Description:
An Intel MIC device runs a Linux OS during its operation. For
file of the card OS.
What: /sys/class/mic/mic(x)/log_buf_len
-Date: August 2013
-KernelVersion: 3.11
+Date: October 2013
+KernelVersion: 3.13
Contact: Sudeep Dutt <sudeep.dutt@intel.com>
Description:
An Intel MIC device runs a Linux OS during its operation. For
implements the three required standard address spaces i.e. configuration,
memory and I/O. The host OS loads a device driver as is typical for
PCIe devices. The card itself runs a bootstrap after reset that
-transfers control to the card OS downloaded from the host driver.
+transfers control to the card OS downloaded from the host driver. The
+host driver supports OSPM suspend and resume operations. It shuts down
+the card during suspend and reboots the card OS during resume.
The card OS as shipped by Intel is a Linux kernel with modifications
for the X100 devices.
goto retry;
mpsslog("%s: %s %d state %s\n",
mic->name, __func__, __LINE__, state);
- if ((!strcmp(state, "offline"))) {
+
+ /*
+ * If the shutdown was initiated by OSPM, the state stays
+ * in "suspended" which is also a valid condition for reset.
+ */
+ if ((!strcmp(state, "offline")) ||
+ (!strcmp(state, "suspended"))) {
free(state);
break;
}
return MIC_SHUTTING_DOWN;
if (!strcmp(state, "reset_failed"))
return MIC_RESET_FAILED;
+ if (!strcmp(state, "suspending"))
+ return MIC_SUSPENDING;
+ if (!strcmp(state, "suspended"))
+ return MIC_SUSPENDED;
mpsslog("%s: BUG invalid state %s\n", mic->name, state);
/* Invalid state */
assert(0);
case MIC_SHUTTING_DOWN:
mic_handle_shutdown(mic);
goto close_error;
+ case MIC_SUSPENDING:
+ mic->boot_on_resume = 1;
+ setsysfs(mic->name, "state", "suspend");
+ mic_handle_shutdown(mic);
+ goto close_error;
+ case MIC_OFFLINE:
+ if (mic->boot_on_resume) {
+ setsysfs(mic->name, "state", "boot");
+ mic->boot_on_resume = 0;
+ }
+ break;
default:
break;
}
while ((file = readdir(dp)) != NULL) {
if (!strncmp(file->d_name, "mic", 3)) {
- mic->next = malloc(sizeof(struct mic_info));
+ mic->next = calloc(1, sizeof(struct mic_info));
if (mic->next) {
mic = mic->next;
- mic->next = NULL;
- memset(mic, 0, sizeof(struct mic_info));
mic->id = atoi(&file->d_name[3]);
mic->name = malloc(strlen(file->d_name) + 16);
if (mic->name)
struct mic_net_info mic_net;
struct mic_virtblk_info mic_virtblk;
int restart;
+ int boot_on_resume;
struct mic_info *next;
};
#define MIC_RESET_TO (45)
+ INIT_COMPLETION(mdev->reset_wait);
mdev->ops->reset_fw_ready(mdev);
mdev->ops->reset(mdev);
for (i = 0; i < MIC_RESET_TO; i++) {
if (mdev->ops->is_fw_ready(mdev))
- return;
+ goto done;
/*
* Resets typically take 10s of seconds to complete.
* Since an MMIO read is required to check if the
msleep(1000);
}
mic_set_state(mdev, MIC_RESET_FAILED);
+done:
+ complete_all(&mdev->reset_wait);
}
/* Initialize the MIC bootparams */
if (MIC_RESET_FAILED == mdev->state)
goto unlock;
mic_set_shutdown_status(mdev, MIC_NOP);
- mic_set_state(mdev, MIC_OFFLINE);
+ if (MIC_SUSPENDED != mdev->state)
+ mic_set_state(mdev, MIC_OFFLINE);
}
unlock:
mutex_unlock(&mdev->mic_mutex);
mutex_lock(&mdev->mic_mutex);
mic_set_shutdown_status(mdev, bootparam->shutdown_status);
bootparam->shutdown_status = 0;
- if (MIC_SHUTTING_DOWN != mdev->state)
+
+ /*
+ * if state is MIC_SUSPENDED, OSPM suspend is in progress. We do not
+ * change the state here so as to prevent users from booting the card
+ * during and after the suspend operation.
+ */
+ if (MIC_SHUTTING_DOWN != mdev->state &&
+ MIC_SUSPENDED != mdev->state)
mic_set_state(mdev, MIC_SHUTTING_DOWN);
mutex_unlock(&mdev->mic_mutex);
}
mic_stop(mdev, false);
}
+
+/**
+ * mic_complete_resume - Complete MIC Resume after an OSPM suspend/hibernate
+ * event.
+ * @mdev: pointer to mic_device instance
+ *
+ * RETURNS: None.
+ */
+void mic_complete_resume(struct mic_device *mdev)
+{
+ if (mdev->state != MIC_SUSPENDED) {
+ dev_warn(mdev->sdev->parent, "state %d should be %d\n",
+ mdev->state, MIC_SUSPENDED);
+ return;
+ }
+
+ /* Make sure firmware is ready */
+ if (!mdev->ops->is_fw_ready(mdev))
+ mic_stop(mdev, true);
+
+ mutex_lock(&mdev->mic_mutex);
+ mic_set_state(mdev, MIC_OFFLINE);
+ mutex_unlock(&mdev->mic_mutex);
+}
+
+/**
+ * mic_prepare_suspend - Handle suspend notification for the MIC device.
+ * @mdev: pointer to mic_device instance
+ *
+ * RETURNS: None.
+ */
+void mic_prepare_suspend(struct mic_device *mdev)
+{
+ int rc;
+
+#define MIC_SUSPEND_TIMEOUT (60 * HZ)
+
+ mutex_lock(&mdev->mic_mutex);
+ switch (mdev->state) {
+ case MIC_OFFLINE:
+ /*
+ * Card is already offline. Set state to MIC_SUSPENDED
+ * to prevent users from booting the card.
+ */
+ mic_set_state(mdev, MIC_SUSPENDED);
+ mutex_unlock(&mdev->mic_mutex);
+ break;
+ case MIC_ONLINE:
+ /*
+ * Card is online. Set state to MIC_SUSPENDING and notify
+ * MIC user space daemon which will issue card
+ * shutdown and reset.
+ */
+ mic_set_state(mdev, MIC_SUSPENDING);
+ mutex_unlock(&mdev->mic_mutex);
+ rc = wait_for_completion_timeout(&mdev->reset_wait,
+ MIC_SUSPEND_TIMEOUT);
+ /* Force reset the card if the shutdown completion timed out */
+ if (!rc) {
+ mutex_lock(&mdev->mic_mutex);
+ mic_set_state(mdev, MIC_SUSPENDED);
+ mutex_unlock(&mdev->mic_mutex);
+ mic_stop(mdev, true);
+ }
+ break;
+ case MIC_SHUTTING_DOWN:
+ /*
+ * Card is shutting down. Set state to MIC_SUSPENDED
+ * to prevent further boot of the card.
+ */
+ mic_set_state(mdev, MIC_SUSPENDED);
+ mutex_unlock(&mdev->mic_mutex);
+ rc = wait_for_completion_timeout(&mdev->reset_wait,
+ MIC_SUSPEND_TIMEOUT);
+ /* Force reset the card if the shutdown completion timed out */
+ if (!rc)
+ mic_stop(mdev, true);
+ break;
+ default:
+ mutex_unlock(&mdev->mic_mutex);
+ break;
+ }
+}
+
+/**
+ * mic_suspend - Initiate MIC suspend. Suspend merely issues card shutdown.
+ * @mdev: pointer to mic_device instance
+ *
+ * RETURNS: None.
+ */
+void mic_suspend(struct mic_device *mdev)
+{
+ struct mic_bootparam *bootparam = mdev->dp;
+ s8 db = bootparam->h2c_shutdown_db;
+
+ mutex_lock(&mdev->mic_mutex);
+ if (MIC_SUSPENDING == mdev->state && db != -1) {
+ bootparam->shutdown_card = 1;
+ mdev->ops->send_intr(mdev, db);
+ mic_set_state(mdev, MIC_SUSPENDED);
+ }
+ mutex_unlock(&mdev->mic_mutex);
+}
#include <linux/cdev.h>
#include <linux/idr.h>
+#include <linux/notifier.h>
#include "mic_intr.h"
* @state: MIC state.
* @shutdown_status: MIC status reported by card for shutdown/crashes.
* @state_sysfs: Sysfs dirent for notifying ring 3 about MIC state changes.
+ * @reset_wait: Waitqueue for sleeping while reset completes.
* @log_buf_addr: Log buffer address for MIC.
* @log_buf_len: Log buffer length address for MIC.
* @dp: virtio device page
* @shutdown_cookie: shutdown cookie.
* @cdev: Character device for MIC.
* @vdev_list: list of virtio devices.
+ * @pm_notifier: Handles PM notifications from the OS.
*/
struct mic_device {
struct mic_mw mmio;
u8 state;
u8 shutdown_status;
struct sysfs_dirent *state_sysfs;
+ struct completion reset_wait;
void *log_buf_addr;
int *log_buf_len;
void *dp;
struct mic_irq *shutdown_cookie;
struct cdev cdev;
struct list_head vdev_list;
+ struct notifier_block pm_notifier;
};
/**
void mic_delete_debug_dir(struct mic_device *dev);
void __init mic_init_debugfs(void);
void mic_exit_debugfs(void);
+void mic_prepare_suspend(struct mic_device *mdev);
+void mic_complete_resume(struct mic_device *mdev);
+void mic_suspend(struct mic_device *mdev);
#endif
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/poll.h>
+#include <linux/suspend.h>
#include <linux/mic_common.h>
#include "../common/mic_dev.h"
return family;
}
+/**
+* mic_pm_notifier: Notifier callback function that handles
+* PM notifications.
+*
+* @notifier_block: The notifier structure.
+* @pm_event: The event for which the driver was notified.
+* @unused: Meaningless. Always NULL.
+*
+* returns NOTIFY_DONE
+*/
+static int mic_pm_notifier(struct notifier_block *notifier,
+ unsigned long pm_event, void *unused)
+{
+ struct mic_device *mdev = container_of(notifier,
+ struct mic_device, pm_notifier);
+
+ switch (pm_event) {
+ case PM_HIBERNATION_PREPARE:
+ /* Fall through */
+ case PM_SUSPEND_PREPARE:
+ mic_prepare_suspend(mdev);
+ break;
+ case PM_POST_HIBERNATION:
+ /* Fall through */
+ case PM_POST_SUSPEND:
+ /* Fall through */
+ case PM_POST_RESTORE:
+ mic_complete_resume(mdev);
+ break;
+ case PM_RESTORE_PREPARE:
+ break;
+ default:
+ break;
+ }
+ return NOTIFY_DONE;
+}
+
/**
* mic_device_init - Allocates and initializes the MIC device structure
*
*
* returns none.
*/
-static void
+static int
mic_device_init(struct mic_device *mdev, struct pci_dev *pdev)
{
+ int rc;
+
mdev->family = mic_get_family(pdev);
mdev->stepping = pdev->revision;
mic_ops_init(mdev);
mdev->irq_info.next_avail_src = 0;
INIT_WORK(&mdev->reset_trigger_work, mic_reset_trigger_work);
INIT_WORK(&mdev->shutdown_work, mic_shutdown_work);
+ init_completion(&mdev->reset_wait);
INIT_LIST_HEAD(&mdev->vdev_list);
+ mdev->pm_notifier.notifier_call = mic_pm_notifier;
+ rc = register_pm_notifier(&mdev->pm_notifier);
+ if (rc) {
+ dev_err(&pdev->dev, "register_pm_notifier failed rc %d\n",
+ rc);
+ goto register_pm_notifier_fail;
+ }
+ return 0;
+register_pm_notifier_fail:
+ flush_work(&mdev->shutdown_work);
+ flush_work(&mdev->reset_trigger_work);
+ return rc;
}
/**
kfree(mdev->bootmode);
flush_work(&mdev->reset_trigger_work);
flush_work(&mdev->shutdown_work);
+ unregister_pm_notifier(&mdev->pm_notifier);
}
/**
goto ida_fail;
}
- mic_device_init(mdev, pdev);
+ rc = mic_device_init(mdev, pdev);
+ if (rc) {
+ dev_err(&pdev->dev, "mic_device_init failed rc %d\n", rc);
+ goto device_init_fail;
+ }
rc = pci_enable_device(pdev);
if (rc) {
pci_disable_device(pdev);
uninit_device:
mic_device_uninit(mdev);
+device_init_fail:
ida_simple_remove(&g_mic_ida, mdev->id);
ida_fail:
kfree(mdev);
[MIC_ONLINE] = "online",
[MIC_SHUTTING_DOWN] = "shutting_down",
[MIC_RESET_FAILED] = "reset_failed",
+ [MIC_SUSPENDING] = "suspending",
+ [MIC_SUSPENDED] = "suspended",
};
/*
goto done;
}
+ if (sysfs_streq(buf, "suspend")) {
+ mic_suspend(mdev);
+ goto done;
+ }
+
count = -EINVAL;
done:
return count;
MIC_ONLINE,
MIC_SHUTTING_DOWN,
MIC_RESET_FAILED,
+ MIC_SUSPENDING,
+ MIC_SUSPENDED,
MIC_LAST
};