drm/i915: use correct node for handling cache domain eviction

It looks like we were incorrectly comparing vma->node against itself
instead of the target node, when evicting for a node on systems where we
need guard pages between regions with different cache domains. As a
consequence we can end up trying to needlessly evict neighbouring nodes,
even if they have the same cache domain, and if they were pinned we
would fail the eviction.

Fixes: 625d988acc28 ("drm/i915: Extract reserving space in the GTT to a helper")
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/20170306235414.23407-3-matthew.auld@intel.com
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>

drm/i915/selftests: don't leak the gem object

For our fake dma objects we can leak the underlying gem object if we
fail to pin our "backing storage".

[   39.952618] =============================================================================
[   39.952625] BUG mock_object (Tainted: G     U         ): Objects remaining in mock_object on __kmem_cache_shutdown()
[   39.952629] -----------------------------------------------------------------------------

[   39.952633] Disabling lock debugging due to kernel taint
[   39.952635] INFO: Slab 0xffffea00086c6a00 objects=21 used=1 fp=0xffff88021b1abc00 flags=0x5fff8000008100
[   39.952640] CPU: 1 PID: 1258 Comm: drv_selftest Tainted: G    BU          4.10.0+ #46
[   39.952641] Hardware name: Apple Inc. MacBookPro11,1/Mac-189A3D4F975D5FFC, BIOS MBP111.88Z.0138.B17.1602221718 02/22/2016
[   39.952642] Call Trace:
[   39.952648]  dump_stack+0x4d/0x6f
[   39.952651]  slab_err+0x9d/0xb0
[   39.952654]  ? ksm_migrate_page+0xe0/0xe0
[   39.952657]  ? on_each_cpu_cond+0x9a/0xc0
[   39.952658]  ? __kmalloc+0x1af/0x1c0
[   39.952660]  ? __kmem_cache_shutdown+0x173/0x3e0
[   39.952661]  __kmem_cache_shutdown+0x196/0x3e0
[   39.952664]  kmem_cache_destroy+0xa0/0x150
[   39.952708]  mock_device_release+0x113/0x140 [i915]
[   39.952726]  drm_dev_release+0x20/0x40 [drm]
[   39.952735]  drm_dev_unref+0x23/0x30 [drm]
[   39.952768]  i915_gem_gtt_mock_selftests+0x55/0x70 [i915]
[   39.952803]  __run_selftests+0x169/0x1c0 [i915]
[   39.952805]  ? 0xffffffffa0151000
[   39.952840]  i915_mock_selftests+0x30/0x60 [i915]
[   39.952869]  i915_init+0xc/0x78 [i915]
[   39.952870]  ? 0xffffffffa0151000
[   39.952872]  do_one_initcall+0x43/0x170
[   39.952874]  ? __vunmap+0x81/0xd0
[   39.952875]  ? kmem_cache_alloc_trace+0x37/0x170
[   39.952877]  ? do_init_module+0x27/0x1f8
[   39.952879]  do_init_module+0x5f/0x1f8
[   39.952881]  load_module+0x2423/0x29b0
[   39.952882]  ? __symbol_put+0x40/0x40
[   39.952885]  ? kernel_read_file+0x1a3/0x1c0
[   39.952887]  SYSC_finit_module+0xbc/0xf0
[   39.952889]  SyS_finit_module+0xe/0x10
[   39.952892]  entry_SYSCALL_64_fastpath+0x13/0x94

v2: use onion teardown and favour i915_gem_object_put

Fixes: 8d28ba4568f4 ("drm/i915: Exercise filling the top/bottom portions of the ppgtt")
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/20170306235414.23407-2-matthew.auld@intel.com
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>

drm/i915/userptr: Disallow wrapping GTT into a userptr

If we allow the user to convert a GTT mmap address into a userptr, we
may end up in recursion hell, where currently we hit a mutex deadlock
but other possibilities include use-after-free during the
unbind/cancel_userptr.

[  143.203989] gem_userptr_bli D    0   902    898 0x00000000
[  143.204054] Call Trace:
[  143.204137]  __schedule+0x511/0x1180
[  143.204195]  ? pci_mmcfg_check_reserved+0xc0/0xc0
[  143.204274]  schedule+0x57/0xe0
[  143.204327]  schedule_timeout+0x383/0x670
[  143.204374]  ? trace_hardirqs_on_caller+0x187/0x280
[  143.204457]  ? trace_hardirqs_on_thunk+0x1a/0x1c
[  143.204507]  ? usleep_range+0x110/0x110
[  143.204657]  ? irq_exit+0x89/0x100
[  143.204710]  ? retint_kernel+0x2d/0x2d
[  143.204794]  ? trace_hardirqs_on_caller+0x187/0x280
[  143.204857]  ? _raw_spin_unlock_irq+0x33/0x60
[  143.204944]  wait_for_common+0x1f0/0x2f0
[  143.205006]  ? out_of_line_wait_on_atomic_t+0x170/0x170
[  143.205103]  ? wake_up_q+0xa0/0xa0
[  143.205159]  ? flush_workqueue_prep_pwqs+0x15a/0x2c0
[  143.205237]  wait_for_completion+0x1d/0x20
[  143.205292]  flush_workqueue+0x2e9/0xbb0
[  143.205339]  ? flush_workqueue+0x163/0xbb0
[  143.205418]  ? __schedule+0x533/0x1180
[  143.205498]  ? check_flush_dependency+0x1a0/0x1a0
[  143.205681]  i915_gem_userptr_mn_invalidate_range_start+0x1c7/0x270 [i915]
[  143.205865]  ? i915_gem_userptr_dmabuf_export+0x40/0x40 [i915]
[  143.205955]  __mmu_notifier_invalidate_range_start+0xc6/0x120
[  143.206044]  ? __mmu_notifier_invalidate_range_start+0x51/0x120
[  143.206123]  zap_page_range_single+0x1c7/0x1f0
[  143.206171]  ? unmap_single_vma+0x160/0x160
[  143.206260]  ? unmap_mapping_range+0xa9/0x1b0
[  143.206308]  ? vma_interval_tree_subtree_search+0x75/0xd0
[  143.206397]  unmap_mapping_range+0x18f/0x1b0
[  143.206444]  ? zap_vma_ptes+0x70/0x70
[  143.206524]  ? __pm_runtime_resume+0x67/0xa0
[  143.206723]  i915_gem_release_mmap+0x1ba/0x1c0 [i915]
[  143.206846]  i915_vma_unbind+0x5c2/0x690 [i915]
[  143.206925]  ? __lock_is_held+0x52/0x100
[  143.207076]  i915_gem_object_set_tiling+0x1db/0x650 [i915]
[  143.207236]  i915_gem_set_tiling_ioctl+0x1d3/0x3b0 [i915]
[  143.207377]  ? i915_gem_set_tiling_ioctl+0x5/0x3b0 [i915]
[  143.207457]  drm_ioctl+0x36c/0x670
[  143.207535]  ? debug_lockdep_rcu_enabled.part.0+0x1a/0x30
[  143.207730]  ? i915_gem_object_set_tiling+0x650/0x650 [i915]
[  143.207793]  ? drm_getunique+0x120/0x120
[  143.207875]  ? __handle_mm_fault+0x996/0x14a0
[  143.207939]  ? vm_insert_page+0x340/0x340
[  143.208028]  ? up_write+0x28/0x50
[  143.208086]  ? vm_mmap_pgoff+0x160/0x190
[  143.208163]  do_vfs_ioctl+0x12c/0xa60
[  143.208218]  ? debug_lockdep_rcu_enabled+0x35/0x40
[  143.208267]  ? ioctl_preallocate+0x150/0x150
[  143.208353]  ? __do_page_fault+0x36a/0x6e0
[  143.208400]  ? mark_held_locks+0x23/0xc0
[  143.208479]  ? up_read+0x1f/0x40
[  143.208526]  ? entry_SYSCALL_64_fastpath+0x5/0xc6
[  143.208669]  ? __fget_light+0xa7/0xc0
[  143.208747]  SyS_ioctl+0x41/0x70

To prevent the possibility of a deadlock, we defer scheduling the worker
until after we have proven that given the current mm, the userptr range
does not overlap a GGTT mmaping. If another thread tries to remap the
GGTT over the userptr before the worker is scheduled, it will be stopped
by its invalidate-range flushing the current work, before the deadlock
can occur.

v2: Improve discussion of how we end up in the deadlock.
v3: Don't forget to mark the userptr as active after a successful
gup_fast. Rename overlaps_ggtt to noncontiguous_or_overlaps_ggtt.
v4: Fix test ordering between invalid GTT mmaping and range completion
(Tvrtko)

Reported-by: Michał Winiarski <michal.winiarski@intel.com>
Testcase: igt/gem_userptr_blits/map-fixed-invalidate-gup
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Michał Winiarski <michal.winiarski@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170308215903.24171-1-chris@chris-wilson.co.uk
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

drm/i915/userptr: Only flush the workqueue if required

To avoid waiting for work from other invalidate-range threads where
not required, only wait on the userptr cancel workqueue if we have added
some work to it.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Michał Winiarski <michal.winiarski@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170307205851.32578-2-chris@chris-wilson.co.uk
Reviewed-by: Michał Winiarski <michal.winiarski@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

drm/i915/userptr: Deactivate a failed userptr if the worker reports an EFAULT

If the worker fails, it no longer has pages to release and can be
immediately removed from the invalidate-tree.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Michał Winiarski <michal.winiarski@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170307205851.32578-1-chris@chris-wilson.co.uk
Reviewed-by: Michał Winiarski <michal.winiarski@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

drm/i915: Fix up verify_encoder_state

The trouble here is that looking at all connector->state in the
verifier isn't good, because that's run from the commit work, which
doesn't hold the connection_mutex. Which means we're only allowed to
look at states in our atomic update.

The simple fix for future proofing would be to switch over to
drm_for_each_connector_in_state, but that has the problem that the
verification then fails if not all connectors are in the state. And we
also need to be careful to check both old and new encoders, and not
screw things up when an encoder gets reassigned.

Note that this isn't the full fix, since we still look at
connector->state. To fix that, we need Maarten's patch series to
switch over to state pointers within drm_atomic_state, but that's a
different series.

v2: Use oldnew iterator (Maarten).

v3: Rebase onto the iter_get/put->iter_begin/end rename.

Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170301095226.30584-6-daniel.vetter@ffwll.ch

drm/i915: use for_each_intel_connector_iter in intel_display.c

This gets rid of the last users of for_each_intel_connector(), remove
that too.

At first I wasn't sure whether the 2 loops in the modeset state
checker should instead only loop over the connectors in the atomic
commit. But we never add connectors to an atomic update if they don't
(or won't have) a CRTC assigned, which means there'd be a gap in check
coverage. Hence loop over everything on those too.

v2: Rebase onto the iter_get/put->iter_begin/end rename.

Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170301095226.30584-5-daniel.vetter@ffwll.ch

drm/i915: Make intel_get_pipe_from_connector atomic

Drive-by fixup while looking at all the connector_list walkers -
holding connection_mutex does actually _not_ give you locking to look
at the legacy drm_connector->encoder->crtc pointer chain. That one is
solely owned by the atomic commit workers. Instead we must inspect the
atomic state.

Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170301095226.30584-4-daniel.vetter@ffwll.ch

drm/i915: use drm_connector_list_iter in intel_opregion.c

One case where I nuked a now unecessary locking, otherwise all just
boring stuff.

v2: Rebase onto the iter_get/put->iter_begin/end rename.

Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170301095226.30584-3-daniel.vetter@ffwll.ch

drm/i915: use drm_connector_list_iter in intel_hotplug.c

Nothing special, just rote conversion.

v2: Rebase onto the iter_get/put->iter_begin/end rename.

Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170301095226.30584-2-daniel.vetter@ffwll.ch

drm/i915: Use drm_connector_list_iter in debugfs

While at it also try to reduce the locking a bit to what's really just
needed instead of everything that we could possibly lock.

Added a new for_each_intel_connector_iter which includes the cast to
intel_connector.

Otherwise just plain transformation with nothing special going on.

v2: Review from Maarten:
- Stick with modeset_lock_all in sink_crc, it looks at crtc->state.
- Fix up early loop exit in i915_displayport_test_active_write.

v3: Rebase onto the iter_get/put->iter_begin/end rename.

Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> (v2)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170301095226.30584-1-daniel.vetter@ffwll.ch

drm/i915: Check for an invalid seqno before __i915_gem_request_started

__i915_gem_request_started() asserts that the seqno is valid, but
i915_spin_request() was not checking before querying whether the request
had started.

Reported-by: Michał Winiarski <michal.winiarski@intel.com>
Fixes: 754c9fd57649 ("drm/i915: Protect the request->global_seqno with the engine->timeline lock")
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Michał Winiarski <michal.winiarski@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170308142238.22994-1-chris@chris-wilson.co.uk
Reviewed-by: Michał Winiarski <michal.winiarski@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

drm/i915: Purge i915_gem_object_is_dead()

i915_gem_object_is_dead() was a temporary lockdep aide whilst
transitioning to a new locking structure for obj->mm. Since commit
1233e2db199d ("drm/i915: Move object backing storage manipulation to its
own locking") it is now unused and should be removed.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170308132629.7987-2-chris@chris-wilson.co.uk

drm/i915: Avoiding recursing on ww_mutex inside shrinker

We have to avoid taking ww_mutex inside the shrinker as we use it as a
plain mutex type and so need to avoid recursive deadlocks:

[  602.771969] =================================
[  602.771970] [ INFO: inconsistent lock state ]
[  602.771973] 4.10.0gpudebug+ #122 Not tainted
[  602.771974] ---------------------------------
[  602.771975] inconsistent {RECLAIM_FS-ON-W} -> {IN-RECLAIM_FS-W} usage.
[  602.771978] kswapd0/40 [HC0[0]:SC0[0]:HE1:SE1] takes:
[  602.771979]  (reservation_ww_class_mutex){+.+.?.}, at: [<ffffffffa054680a>] i915_gem_object_wait+0x39a/0x410 [i915]
[  602.772020] {RECLAIM_FS-ON-W} state was registered at:
[  602.772024]   mark_held_locks+0x76/0x90
[  602.772026]   lockdep_trace_alloc+0xb8/0xc0
[  602.772028]   __kmalloc_track_caller+0x5d/0x130
[  602.772031]   krealloc+0x89/0xb0
[  602.772033]   reservation_object_reserve_shared+0xaf/0xd0
[  602.772055]   i915_gem_do_execbuffer.isra.35+0x1413/0x18b0 [i915]
[  602.772075]   i915_gem_execbuffer2+0x10e/0x1d0 [i915]
[  602.772078]   drm_ioctl+0x291/0x480
[  602.772079]   do_vfs_ioctl+0x695/0x6f0
[  602.772081]   SyS_ioctl+0x3c/0x70
[  602.772084]   entry_SYSCALL_64_fastpath+0x18/0xad
[  602.772085] irq event stamp: 5197423
[  602.772088] hardirqs last  enabled at (5197423): [<ffffffff8116751d>] kfree+0xdd/0x170
[  602.772091] hardirqs last disabled at (5197422): [<ffffffff811674f9>] kfree+0xb9/0x170
[  602.772095] softirqs last  enabled at (5190992): [<ffffffff8107bfe1>] __do_softirq+0x221/0x280
[  602.772097] softirqs last disabled at (5190575): [<ffffffff8107c294>] irq_exit+0x64/0xc0
[  602.772099]
               other info that might help us debug this:
[  602.772100]  Possible unsafe locking scenario:

[  602.772101]        CPU0
[  602.772101]        ----
[  602.772102]   lock(reservation_ww_class_mutex);
[  602.772104]   <Interrupt>
[  602.772105]     lock(reservation_ww_class_mutex);
[  602.772107]
                *** DEADLOCK ***

[  602.772109] 2 locks held by kswapd0/40:
[  602.772110]  #0:  (shrinker_rwsem){++++..}, at: [<ffffffff811337b5>] shrink_slab.constprop.62+0x35/0x280
[  602.772116]  #1:  (&dev->struct_mutex){+.+.+.}, at: [<ffffffffa0553957>] i915_gem_shrinker_lock+0x27/0x60 [i915]
[  602.772141]
               stack backtrace:
[  602.772144] CPU: 2 PID: 40 Comm: kswapd0 Not tainted 4.10.0gpudebug+ #122
[  602.772145] Hardware name: LENOVO 42433ZG/42433ZG, BIOS 8AET64WW (1.44 ) 07/26/2013
[  602.772147] Call Trace:
[  602.772151]  dump_stack+0x68/0xa1
[  602.772153]  print_usage_bug+0x1d4/0x1f0
[  602.772155]  mark_lock+0x390/0x530
[  602.772157]  ? print_irq_inversion_bug+0x200/0x200
[  602.772159]  __lock_acquire+0x405/0x1260
[  602.772181]  ? i915_gem_object_wait+0x39a/0x410 [i915]
[  602.772183]  lock_acquire+0x60/0x80
[  602.772205]  ? i915_gem_object_wait+0x39a/0x410 [i915]
[  602.772207]  mutex_lock_nested+0x69/0x760
[  602.772229]  ? i915_gem_object_wait+0x39a/0x410 [i915]
[  602.772231]  ? kfree+0xdd/0x170
[  602.772253]  ? i915_gem_object_wait+0x163/0x410 [i915]
[  602.772255]  ? trace_hardirqs_on_caller+0x18d/0x1c0
[  602.772256]  ? trace_hardirqs_on+0xd/0x10
[  602.772278]  i915_gem_object_wait+0x39a/0x410 [i915]
[  602.772300]  i915_gem_object_unbind+0x5e/0x130 [i915]
[  602.772323]  i915_gem_shrink+0x22d/0x3d0 [i915]
[  602.772347]  i915_gem_shrinker_scan+0x3f/0x80 [i915]
[  602.772349]  shrink_slab.constprop.62+0x1ad/0x280
[  602.772352]  shrink_node+0x52/0x80
[  602.772355]  kswapd+0x427/0x5c0
[  602.772358]  kthread+0x122/0x130
[  602.772360]  ? try_to_free_pages+0x270/0x270
[  602.772362]  ? kthread_stop+0x70/0x70
[  602.772365]  ret_from_fork+0x2e/0x40

v2: Add commentary about the pruning being opportunistic

Reported-by: Jan Nordholz <jckn@gmx.net>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99977#c10
Fixes: e54ca9774777 ("drm/i915: Remove completed fences after a wait")
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170308132629.7987-1-chris@chris-wilson.co.uk

drm/i915: Use DRM_DEBUG_KMS() for framebuffer failure debug messages

DRM_UT_CORE generates way too much noise usually, so having the
framebuffer init failures use DRM_UT_CORE is a pain when trying to
find out the reason why you failed in creating a framebuffer.
Let's use DRM_UT_KMS for these debug messages instead.

v2: s/at less than/at most/ in the debug message (Imre)

Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: Imre Deak <imre.deak@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170307194210.13400-6-ville.syrjala@linux.intel.com

drm/i915: Pass the correct plane index to _intel_compute_tile_offset()

intel_fill_fb_info() should pass the correct plane index to
_intel_compute_tile_offset() once we start to care about the AUX
surface.

Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: Imre Deak <imre.deak@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170307194210.13400-5-ville.syrjala@linux.intel.com

drm/i915: Avoid div-by-zero when computing aux_stride w/o an aux plane

To make life easier let's allow skl_plane_stride() to be called for the
AUX surface even when there is no AUX surface. Avoids special cases in
the callers.

Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: Imre Deak <imre.deak@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170307194210.13400-4-ville.syrjala@linux.intel.com

drm/i915: Move nv12 chroma plane handling into intel_surf_alignment()

Let's try to keep the alignment requirements in one place, and so
towards that end let's move the AUX_DIST alignment handling into
intel_surf_alignment() alongside the main surface alignment stuff.

Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: Imre Deak <imre.deak@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170307194210.13400-3-ville.syrjala@linux.intel.com

drm/i915: Plumb drm_framebuffer into more places

Now that framebuffers can be used even before calling
drm_framebuffer_init() we can start to plumb them into more places,
instead of passing individual pieces for fb metadata.

Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: Imre Deak <imre.deak@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170307194210.13400-2-ville.syrjala@linux.intel.com

drm/i915: suppress atomic commit error message under gvt-g env

under virtualization enviroment, it is possible guest update pipe
registers across vblank intervals due to overhead of mmio traps or vm
schedule out. However, it is safe since those pipe update happen in
virual registers and will not be committed to hardware. suppress that
atomic commit error message under virtualization case to avoid
confusing user.

v2: per ville's comment: return early and against Maarten's patch
v3: coding style clean

Signed-off-by: Bing Niu <bing.niu@intel.com>
Acked-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1489004043-15449-1-git-send-email-bing.niu@intel.com
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>

drm/i915: Nuke debug messages from the pipe update critical section

printks are slow so we should not be doing them from the vblank evade
critical section. These could explain why we sometimes seem to
blow past our 100 usec deadline.

The problem has been there ever since commit bfd16b2a23dc ("drm/i915:
Make updating pipe without modeset atomic.") but it may not have
been readily visible until commit e1edbd44e23b ("drm/i915: Complain
if we take too long under vblank evasion.") increased our chances
of noticing it.

Cc: stable@vger.kernel.org
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Fixes: bfd16b2a23dc ("drm/i915: Make updating pipe without modeset atomic.")
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170307205419.19447-1-ville.syrjala@linux.intel.com
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>

Merge remote-tracking branch 'airlied/drm-next' into drm-intel-next-queued

Backmerge drm-next to get at all the good stuff in drm-misc. We need
that because:

- drm_connector_list_iter conversion for i915 needs the core patches.
- Maarten's patches to use the new atomic state iterators also need
  the core patches.
- We need the new link status property to complete the DP retraining
  work, merging through 2 branches wasn't a good idea and we had to
  partially backtrack.
- Chris needs reservation_object_trylock and we want to roll out
  kref_read everywhere.

Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>

Merge branch 'linux-4.12' of git://github.com/skeggsb/linux into drm-next

- Re-architecture of the code to handle proprietary fw, more abstracted
to support the multitude of differences that NVIDIA introduce
- Support in the said code for GP10x ACR and GR fw, giving acceleration
support \o/
- Fix for GTX 970 GPUs that are in an odd MMU configuration

* 'linux-4.12' of git://github.com/skeggsb/linux: (60 commits)
  drm/nouveau/fb/gf100-: rework ram detection
  drm/nouveau/fb/gm200: split ram implementation from gm107
  drm/nouveau/fb/gf108: split implementation from gf100
  drm/nouveau/fb/gf100-: modify constructors to allow more customisation
  drm/nouveau/kms/nv50: use drm core i2c-over-aux algorithm
  drm/nouveau/i2c/g94-: return REPLY_M value on reads
  drm/nouveau/i2c: modify aux interface to return length actually transferred
  drm/nouveau/gp10x: enable secboot and GR
  drm/nouveau/gr/gp102: initial support
  drm/nouveau/falcon: support for gp10x msgqueue
  drm/nouveau/secboot: add gp102/gp104/gp106/gp107 support
  drm/nouveau/secboot: put HS code loading code into own file
  drm/nouveau/secboot: support for r375 ACR
  drm/nouveau/secboot: support for r367 ACR
  drm/nouveau/secboot: support for r364 ACR
  drm/nouveau/secboot: workaround bug when starting SEC2 firmware
  drm/nouveau/secboot: support standard NVIDIA HS binaries
  drm/nouveau/secboot: support for unload blob bootloader
  drm/nouveau/secboot: let callers interpret return value of blobs
  drm/nouveau/secboot: support for different load and unload falcons
  ...

Merge tag 'drm-intel-next-2017-03-06' of git://anongit.freedesktop.org/git/drm-intel into drm-next

4 weeks worth of stuff since I was traveling&lazy:

- lspcon improvements (Imre)
- proper atomic state for cdclk handling (Ville)
- gpu reset improvements (Chris)
- lots and lots of polish around fences, requests, waiting and
  everything related all over (both gem and modeset code), from Chris
- atomic by default on gen5+ minus byt/bsw (Maarten did the patch to
  flip the default, really this is a massive joint team effort)
- moar power domains, now 64bit (Ander)
- big pile of in-kernel unit tests for various gem subsystems (Chris),
  including simple mock objects for i915 device and and the ggtt
  manager.
- i915_gpu_info in debugfs, for taking a snapshot of the current gpu
  state. Same thing as i915_error_state, but useful if the kernel didn't
  notice something is stick. From Chris.
- bxt dsi fixes (Umar Shankar)
- bxt w/a updates (Jani)
- no more struct_mutex for gem object unreference (Chris)
- some execlist refactoring (Tvrtko)
- color manager support for glk (Ander)
- improve the power-well sync code to better take over from the
  firmware (Imre)
- gem tracepoint polish (Tvrtko)
- lots of glk fixes all around (Ander)
- ctx switch improvements (Chris)
- glk dsi support&fixes (Deepak M)
- dsi fixes for vlv and clanups, lots of them (Hans de Goede)
- switch to i915.ko types in lots of our internal modeset code (Ander)
- byt/bsw atomic wm update code, yay (Ville)

* tag 'drm-intel-next-2017-03-06' of git://anongit.freedesktop.org/git/drm-intel: (432 commits)
  drm/i915: Update DRIVER_DATE to 20170306
  drm/i915: Don't use enums for hardware engine id
  drm/i915: Split breadcrumbs spinlock into two
  drm/i915: Refactor wakeup of the next breadcrumb waiter
  drm/i915: Take reference for signaling the request from hardirq
  drm/i915: Add FIFO underrun tracepoints
  drm/i915: Add cxsr toggle tracepoint
  drm/i915: Add VLV/CHV watermark/FIFO programming tracepoints
  drm/i915: Add plane update/disable tracepoints
  drm/i915: Kill level 0 wm hack for VLV/CHV
  drm/i915: Workaround VLV/CHV sprite1->sprite0 enable underrun
  drm/i915: Sanitize VLV/CHV watermarks properly
  drm/i915: Only use update_wm_{pre,post} for pre-ilk platforms
  drm/i915: Nuke crtc->wm.cxsr_allowed
  drm/i915: Compute proper intermediate wms for vlv/cvh
  drm/i915: Skip useless watermark/FIFO related work on VLV/CHV when not needed
  drm/i915: Compute vlv/chv wms the atomic way
  drm/i915: Compute VLV/CHV FIFO sizes based on the PM2 watermarks
  drm/i915: Plop vlv/chv fifo sizes into crtc state
  drm/i915: Plop vlv wm state into crtc_state
  ...

drm/i915: Use pagecache write to prepopulate shmemfs from pwrite-ioctl

Before we instantiate/pin the backing store for our use, we
can prepopulate the shmemfs filp efficiently using a write into the
pagecache. We avoid the penalty of instantiating all the pages, important
if the user is just writing to a few and never uses the object on the GPU,
and using a direct write into shmemfs allows it to avoid the cost of
retrieving a page (mostly the clear-before-use, but in theory we could
curtail swapin) before it is overwritten.

This can be extended later to provide additional specialisation for
other backends (other than shmemfs). For now it provides a defense
against very large write-only allocations from exhausting all of system
memory.

v2: Smelling fixes.

Fixes: fe115628d567 ("drm/i915: Implement pwrite without struct-mutex")
References: https://bugs.freedesktop.org/show_bug.cgi?id=99107
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Matthew Auld <matthew.william.auld@gmail.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: <stable@vger.kernel.org> # v4.10+
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170307120338.7277-2-chris@chris-wilson.co.uk

drm/i915: Store a permanent error in obj->mm.pages

Once the object has been truncated, it is unrecoverable. To facilitate
detection of this state store the error in obj->mm.pages.

This is required for the next patch which should be applied to v4.10
(via stable), so we also need to mark this patch for backporting. In
that regard, let's consider this to be a fix/improvement too.

v2: Avoid dereferencing the ERR_PTR when freeing the object.

Fixes: 1233e2db199d ("drm/i915: Move object backing storage manipulation to its own locking")
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: <stable@vger.kernel.org> # v4.10+
Link: http://patchwork.freedesktop.org/patch/msgid/20170307132031.32461-1-chris@chris-wilson.co.uk
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>

drm/i915: move the {skl, bxt}_{i, uni}nit_cdclk declarations

Move the {skl,bxt}_{i,uni}nit_cdclk declarations to the place where
the intel_cdclk.c functions are declared since these functions have
moved there.

Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: Dhinakaran Pandiyan <dhinakaran.pandiyan@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1487712207-17181-1-git-send-email-paulo.r.zanoni@intel.com

drm/i915: remove potentially confusing IS_G4X checks

The IS_G4X macro is defined as IS_G45 || IS_GM45. We have two points
in our code where we have an if statement checking for GM45 followed
by an else if statement checking for IS_G4X. This can be confusing
since the IS_G4X check won't be catching the previously-checked GM45.
Someone quickly trying to check which functions run on each platform
may end up getting confused while reading the code.

Fix the potential confusion by limiting the else if statements to only
check for the platform that was not already checked earlier in the if
ladder.

Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1487620842-22893-3-git-send-email-paulo.r.zanoni@intel.com

drm/i915: Flush idle work when changing missed-irq fault injection

In order for the missed-irq update to take effect, the device must be
idle. So when the user updates the fault injection via debugfs, idle the
device.

v2: Idle is explicitly required for setting test_irq, and good behaviour
for clearing the missed_irq.
v3: Use matching types; expanding to more than ulong rings is left as an
exercise to the reader.

Testcase: igt/drv_missed_irq
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170307155908.14576-1-chris@chris-wilson.co.uk

drm/i915/dsi: arrange intel_dsi.h according to relevant files

No functional changes.

Reviewed-by: Hans de Goede <hdegoede@redhat.com>
Reviewed-by: Bob Paauwe <bob.j.paauwe@intel.com>
Cc: Madhav Chauhan <madhav.chauhan@intel.com>
Cc: Hans de Goede <hdegoede@redhat.com>
Cc: Bob Paauwe <bob.j.paauwe@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/fadc3b0de70d2c7db7a6ecbe7121328483330f04.1488810382.git.jani.nikula@intel.com

drm/i915/dsi: rename intel_dsi_panel_vbt.c to intel_dsi_vbt.c

Emphasize that the VBT file is nowadays more about initializing and
running stuff based on the VBT contents, not so much about being a
"panel driver". No functional changes.

Reviewed-by: Hans de Goede <hdegoede@redhat.com>
Reviewed-by: Bob Paauwe <bob.j.paauwe@intel.com>
Cc: Madhav Chauhan <madhav.chauhan@intel.com>
Cc: Hans de Goede <hdegoede@redhat.com>
Cc: Bob Paauwe <bob.j.paauwe@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/b13cb012a555ff5eb56b5e4bb2b0205c3e025a99.1488810382.git.jani.nikula@intel.com

drm/i915/dsi: rename intel_dsi_pre_disable to intel_dsi_disable

The hook names reflect more the phase in the mode set sequence the hooks
are called in than what they actually do in terms of the specific
encoder. Stick to that scheme, and rename intel_dsi_pre_disable to
intel_dsi_disable. Unify the comments around this while at it. No
functional changes.

v2: Add more sense in the enable/disable hook comments (Ville)

Reviewed-by: Hans de Goede <hdegoede@redhat.com>
Reviewed-by: Bob Paauwe <bob.j.paauwe@intel.com>
Cc: Madhav Chauhan <madhav.chauhan@intel.com>
Cc: Hans de Goede <hdegoede@redhat.com>
Cc: Bob Paauwe <bob.j.paauwe@intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1488878659-10386-1-git-send-email-jani.nikula@intel.com

drm/i915/dsi: rename intel_dsi_exec_vbt_sequence to intel_dsi_vbt_exec_sequence

Use the prefix intel_dsi_vbt for all the DSI VBT functions. No
functional changes.

Reviewed-by: Hans de Goede <hdegoede@redhat.com>
Reviewed-by: Bob Paauwe <bob.j.paauwe@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Madhav Chauhan <madhav.chauhan@intel.com>
Cc: Hans de Goede <hdegoede@redhat.com>
Cc: Bob Paauwe <bob.j.paauwe@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/0a05abca364f3bc7f9caf90c9bd3a68eef5f222f.1488810382.git.jani.nikula@intel.com

drm/i915/dsi: stop using the drm_panel framework completely

Now that we've stopped using the drm_panel hooks, there aren't any
benefits left with using the drm_panel framework. Remove the rest of the
drm_panel use. No functional changes.

Reviewed-by: Hans de Goede <hdegoede@redhat.com>
Reviewed-by: Bob Paauwe <bob.j.paauwe@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Madhav Chauhan <madhav.chauhan@intel.com>
Cc: Hans de Goede <hdegoede@redhat.com>
Cc: Bob Paauwe <bob.j.paauwe@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/6602e36641451952065092401bd6e6cfbe93e208.1488810382.git.jani.nikula@intel.com

drm/i915/dsi: call vbt_panel_get_modes directly instead of via drm_panel

Commit 18a00095a5f3 ("drm/i915/dsi: Make intel_dsi_enable/disable
directly exec VBT sequences") started calling the VBT sequence functions
directly instead of using the drm_panel hooks. Remove the last drm_panel
hook by calling vbt_panel_get_modes() directly. No functional changes.

Reviewed-by: Hans de Goede <hdegoede@redhat.com>
Reviewed-by: Bob Paauwe <bob.j.paauwe@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Madhav Chauhan <madhav.chauhan@intel.com>
Cc: Hans de Goede <hdegoede@redhat.com>
Cc: Bob Paauwe <bob.j.paauwe@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/63d0d41f29583507f5968b42b5f52e6574a1f245.1488810382.git.jani.nikula@intel.com

drm/i915/dsi: remove support for more than one panel driver

Fact is, there are no other panel drivers except the VBT based
one. Simplify the code and maintenance. No functional changes.

Reviewed-by: Hans de Goede <hdegoede@redhat.com>
Reviewed-by: Bob Paauwe <bob.j.paauwe@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Madhav Chauhan <madhav.chauhan@intel.com>
Cc: Hans de Goede <hdegoede@redhat.com>
Cc: Bob Paauwe <bob.j.paauwe@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/7dfd041dd25e8e930150ede09589bb232f6248d5.1488810382.git.jani.nikula@intel.com

drm/i915: Avoid clearing the base drm_crtc_state

To prevent having to preserve the drm_crtc_state as we clear the
intel_crtc_state, only memset our extended state.

Fixes:
drivers/gpu/drm/i915/intel_display.c: In function ‘clear_intel_crtc_state’:
drivers/gpu/drm/i915/intel_display.c:11301:1: error: the frame size of 1056 bytes is larger than 1024 bytes [-Werror=frame-larger-than=]

v2: Add a comment and BUILD_BUG_ON to explain the memset()

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/20170303154644.6709-1-chris@chris-wilson.co.uk
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>

drm/i915/: DMC 1.04 for Geminilake

There is a nre version of DMC available for GLK.

The release notes mentions:
This FW has the fix to remove the hang conditions due to
some debug related issues.

Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Anusha Srivatsa <anusha.srivatsa@intel.com>
Reviewed-by: Ander Conselvan de Oliveira <conselvan2@gmail.com>
Signed-off-by: Ander Conselvan de Oliveira <ander.conselvan.de.oliveira@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1487793336-31857-1-git-send-email-anusha.srivatsa@intel.com

drm/i915: No need to save/restore irq status in intel_engine_wakeup

It is called from either the process or timer context so it is
correct to always disable interrupts.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/20170306150321.29024-1-tvrtko.ursulin@linux.intel.com

drm/i915: No need to save/restore irq status in intel_breadcrumbs_fake_irq

Timer callback is a known context so it is correct to always
disable interrupts.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>

drm/i915: No need to save/restore irq status in __i915_request_irq_complete

It is always called from thread context.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>

drm/nouveau/fb/gf100-: rework ram detection

This commit reworks the RAM detection algorithm, using RAM-per-LTC to
determine whether a board has a mixed-memory configuration instead of
using RAM-per-FBPA.  I'm not certain the algorithm is perfect, but it
should handle all currently known configurations in the very least.

This should fix GTX 970 boards with 4GiB of RAM where the last 512MiB
isn't fully accessible, as well as only detecting half the VRAM on
GF108 boards.

As a nice side-effect, GP10x memory detection now reuses the majority
of the code from earlier chipsets.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/fb/gm200: split ram implementation from gm107

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/fb/gf108: split implementation from gf100

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/fb/gf100-: modify constructors to allow more customisation

GF108/GM107 implementations will want slightly different functions for
the upcoming RAM detection improvements.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/kms/nv50: use drm core i2c-over-aux algorithm

I'm not entirely sure NVKM needs to support this now, but I haven't
removed it as of yet just in case it's needed from DEVINIT scripts
where DRM isn't available.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/i2c/g94-: return REPLY_M value on reads

This value represents the actual number of bytes recieved on the AUX
channel as the result of a read transaction.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/i2c: modify aux interface to return length actually transferred

Apparently sinks are allows to respond with ACK even if they didn't
fully complete a transaction...  It seems like a missed opportunity
for DEFER to me, but what do I know :)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gp10x: enable secboot and GR

All the bricks are in place for secure boot to be enabled. This in turn
makes GR usable so enable them all.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gp102: initial support

Differences from GP100:
- 3 PPCs/GPC.
- Another random reg to calculate/write.
- Attrib CB setup a little different.
- PascalB
- PascalComputeB

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon: support for gp10x msgqueue

Add support for the msgqueue firmware used to process SEC2 commands
for gp10x chips.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: add gp102/gp104/gp106/gp107 support

These gp10x chips are supporting using (roughly) the same firmware.
Compared to previous secure chips, ACR runs on SEC2 and so does the
low-secure msgqueue.

ACR for these chips is based on r367.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: put HS code loading code into own file

We will also need to load HS blobs outside of acr_r352 (for instance, to
run the NVDEC VPR scrubber), so make this code reusable.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: support for r375 ACR

r375 ACR uses a unified bootloader descriptor for the GR and PMU
firmwares.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: support for r367 ACR

r367 uses a different hsflcn_desc layout and LS firmware signature
format, requiring a rewrite of some functions.

It also makes use of the shadow region, and uses SEC as the boot falcon.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: support for r364 ACR

r364 is similar to r361, but uses a different hsflcn_desc structure to
introduce the shadow region address (even though it is not yet used by
this version).

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: workaround bug when starting SEC2 firmware

For some unknown reason the LS SEC2 firmware needs to be started twice
to operate. Detect and address that condition.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: support standard NVIDIA HS binaries

I had the brilliant idea to "improve" the binary format by removing
a useless indirection in the HS binary files. In the end it just
makes things more complicated than they ought to be as NVIDIA-provided
files need to be adapted. Since the format used can be identified by the
header, support both.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: support for unload blob bootloader

If the load and unload falcons are different, then a different
bootloader must also be used. Support this case.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: let callers interpret return value of blobs

Since the HS blobs are provided and signed by NVIDIA, we cannot expect
always-consistent behavior. In this case, on GP10x the unload blob may
return 0x1d even though things have run perfectly well. This behavior
has been confirmed by NVIDIA.

So let the callers of the run_blob() hook receive the blob return's
value (a positive integer) and decide what it means. This allows us to
workaround the 0x1d code instead of issuing an error.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: support for different load and unload falcons

On some secure boot instances (e.g. gp10x) the load and unload blobs do
not run on the same falcon. Support this case by introducing a new
member to the ACR structure and making related functions take the falcon
to use as an argument instead of assuming the boot falcon is to be used.

The rule is that the load blob can be run on either the SEC or PMU
falcons, but the unload blob must be always run on PMU.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: share r361 BL structures and functions

Share elements of r361 that will be reused in other ACRs.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: add support for SEC LS firmware

Support running a message queue firmware on SEC.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: support running ACR on SEC

Add support for running the ACR binary on the SEC falcon.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: get start address of blob from ACR

The start address used for secure blobs is not unique to the ACR, but
rather blob-dependent. Remove the unique member stored in the ACR
structure and make the load function return the start address for the
current blob instead.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: add shadow blob argument

ACR firmware from r364 on need a shadow region for the ACR to copy the
WPR region into. Add a flag to indicate that a shadow region is required
and manage memory allocations accordingly.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon/msgqueue: add SEC2 support

Add support for running a msgqueue on the SEC2 falcon.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon: support for EMEM

On SEC, DMEM is unaccessible by the CPU when the falcon is running in LS
mode. This makes communication with the firmware using DMEM impossible.

For this purpose, a new kind of memory (EMEM) has been added. It works
similarly to DMEM, with the difference that its address space starts at
0x1000000. For this reason, it makes sense to treat it like a special
case of DMEM.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon: fix base address of FBIF registers

All falcons have their FBIF registers starting at offset 0x600, with the
exception of the PMU and NVENC engines.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon: better detection of debug register

Not all falcons have a debug register, and it is not always found at the
same offset.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/core: add SEC2 engine

SEC2 is the name given by NVIDIA to the SEC engine post-Fermi (reasons
unknown). Even though it shares the same address range as SEC, its usage
is quite different and this justifies a new engine. Add this engine and
make TOP use it all post-TOP devices should use this implementation and
not the older SEC.

Also quickly add the short gp102 implementation which will be used for
falcon booting purposes.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/nvdec: add gp102 support

gp10x' secure boot requires a blob to be run on NVDEC. Expose the falcon
through a dummy device.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon: delay construction of falcons to oneinit()

Reading registers at device construction time can be harmful, as there
is no guarantee the underlying engine will be up, or in its runtime
configuration. Defer register reading to the oneinit() hook and update
users accordingly.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon: use NXTCTX register instead of NEW_INSTBLK

Both registers allow to bind a new context, but NXTCTX will work on all
falcons, while legacy NEW_INSTBLK is reserved to PMU.

After setting NXTCTX we trigger a context switch by writing 0x090 and
0x0a4.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot/gm20b: enable PMU firmware

Enable the PMU firmware in gm20b, managed by secure boot.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/pmu/gm20b: add msgqueue support

gm20b PMU firmware is driven by a msgqueue, so connect relevant PMU
hooks to their msgqueue counterparts.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: check that WPR region is properly set

The ACR firmware may return no error but fail nonetheless. Such cases
can be detected by verifying that the WPR region has been properly set
in FB. If this is not the case, this is an error, but the unload
firmware should still not be run.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: support optional falcons

PMU support has been enabled for r352 ACR, but it must remain optional
if we want to preserve existing user-space that do not include it. Allow
ACR to be instanciated with a list of optional LS falcons, that will not
produce a fatal error if their firmware is not loaded. Also change the
secure boot bootstrap logic to be able to fall back to legacy behavior
if it turns out the boot falcon's LS firmware cannot be loaded.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: support PMU LS firmware

Add the PMU bootloader generator and PMU LS ops that will enable proper
PMU operation if the PMU falcon is designated as managed.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: base support for PMU falcon

Adapt secboot's behavior if a PMU firmware is present, in particular
the way LS falcons are reset. Without PMU firmware, secboot needs to be
performed again from scratch so all LS falcons are reset. With PMU
firmware, we can ask the PMU's ACR unit to reset a specific falcon
through a PMU message.

As we must preserve the old behavior to avoid breaking user-space, add a
few conditionals to the way falcons are reset.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: support for loading LS PMU firmware

Allow secboot to load a LS PMU firmware. LS PMU is one instance of
firmwares based on the message queue mechanism, which is also used for
other firmwares like SEC, so name its source file accordingly.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/pmu: add msgqueue member

NVIDIA-provided PMU firmware is controlled by a msgqueue. Add a member
to the PMU structure as well as the required cleanup code if this
feature is used.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon: support for gm20b msgqueue

Add support for the msgqueue firmware used to process PMU commands for
gm20b.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon: add msgqueue interface

A message queue firmware implements a specific protocol allowing the
host to send "commands" to a falcon, and the falcon to reply using
"messages". This patch implements the common part of this protocol and
defines the interface that the host can use.

Due to the way the firmware is developped internally at NVIDIA (where
kernel driver and firmware evolve in lockstep), firmwares taken at
different points in time can have frustratingly subtle differences that
must be taken into account. This code is architectured to make
implementing such differences as easy as possible.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: add LS firmware post-run hooks

Add the ability for LS firmwares to declare a post-run hook that is
invoked right after the HS firmware is executed. This allows them to
e.g. write some initialization data into the falcon's DMEM.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: abstract fixup_hs_desc function

As different firmare versions use different HS descriptor formats, we
need to abstract this part as well.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: make specialized ls_ucode_img struct private

This structure does not need to be shared anymore.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: store ucode offset in base image structure

This allows the bootloader descriptor generation code to not rely on
specialized ls_ucode_img structures, making it reusable in other
instances.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: fix usage of hsf_load_header

Offsets were not properly computed. This went unnoticed because we are
only using one app for now.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: prevent address trimming

Using 32-bit integers would trim the WPR address if it is allocated above 4GB.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: fix WPR region alignment

A WPR region smaller than 256K will result in secure boot failure.
Adjust the minimal size.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: fix WPR address to be 64-bit

The WPR address parameter of the ls_write_wpr hook was defined as a u32,
which will very likely overflow on boards with more than 4GB VRAM.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: make sure requested falcons are supported

Check at contruction time that we have support for all the LS firmwares
asked by the caller.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: remove unused hook

Remove a leftover that became obsolete with the falcon interface.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon: fix IMEM port access

All IMEM registers are duplicated per port.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon: fix port offset for DMEM register

DMEM registers are replicated with a stride of 8 bytes.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon: protect against concurrent DMEM accesses

The falcon library may be used concurrently, especially after the
introduction of the msgqueue interface. Make it safe to use it that way.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon: add missing context binding memory target

This is not used currently, but is added for the sake of completeness.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/pmu: make sure the reset hook exists before running it

Some PMU implementations (in particular the ones managed by secure
boot) may not have a reset() hook. Make sure we don't crash in that
case.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: make nvkm_secboot_falcon_name visible

Make nvkm_secboot_falcon_name publicly visible as other subdevs will
need to use it for debug messages.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>