drm/amdgpu: Make SDMA phase quantum configurable

Set a configurable SDMA phase quantum when enabling SDMA context
switching. The default value significantly reduces SDMA latency
in page table updates when user-mode SDMA queues have concurrent
activity, compared to the initial HW setting.

Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Andres Rodriguez <andres.rodriguez@amd.com>
Reviewed-by: Shaoyun Liu <shaoyun.liu@amd.com>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Enable SDMA context switching for CIK

Enable SDMA context switching on CIK (copied from sdma_v3_0.c).

Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Enable SDMA_CNTL.ATC_L1_ENABLE for SDMA on CZ

For GFX context, the  ATC bit in SDMA*_GFX_VIRTUAL_ADDRESS  can be cleared
to perform in VM mode. For RLC context, to support ATC mode , ATC bit in
SDMA*_RLC*_VIRTUAL_ADDRESS should be set. SDMA_CNTL.ATC_L1_ENABLE bit is
global setting that enables the  L1-L2 translation for ATC address.

Signed-off-by: shaoyun liu <shaoyun.liu@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Try evicting from CPU visible to invisible VRAM first

This gives BOs which haven't been accessed by the CPU since they were
moved to visible VRAM another chance to stay in VRAM when another BO
needs to go to visible VRAM.

This should allow BOs to stay in VRAM longer in some cases.

v2:
* Only do this for BOs which don't have the
  AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED flag set.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Don't force BOs into visible VRAM for page faults

There is no need for page faults to force BOs into visible VRAM if it's
full, and the time it takes to do so is great enough to cause noticeable
stuttering. Add GTT as a possible placement so that if visible VRAM is
full, page faults move BOs to GTT instead of evicting other BOs from VRAM.

Suggested-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: John Brooks <john@fastquake.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Set/clear CPU_ACCESS flag on page fault and move to VRAM

When a BO is moved to VRAM, clear AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED.
This allows it to potentially later move to invisible VRAM if the CPU
does not access it again.

Setting the CPU_ACCESS flag in amdgpu_bo_fault_reserve_notify() also means
that we can remove the loop to restrict lpfn to the end of visible VRAM,
because amdgpu_ttm_placement_init() will do it for us.

v3 [Michel Dänzer]
* Use AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED instead of a new flag
  (Christian König)
* Clear flag in amdgpu_bo_move instead of amdgpu_move_ram_vram
  (Christian)
* Explicitly mention amdgpu_bo_fault_reserve_notify in amdgpu_bo_move
* Also clear flag in amdgpu_bo_create_restricted

Suggested-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: John Brooks <john@fastquake.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Throttle visible VRAM moves separately

The BO move throttling code is designed to allow VRAM to fill quickly if it
is relatively empty. However, this does not take into account situations
where the visible VRAM is smaller than total VRAM, and total VRAM may not
be close to full but the visible VRAM segment is under pressure. In such
situations, visible VRAM would experience unrestricted swapping and
performance would drop.

Add a separate counter specifically for moves involving visible VRAM, and
check it before moving BOs there.

v2: Only perform calculations for separate counter if visible VRAM is
    smaller than total VRAM. (Michel Dänzer)
v3: [Michel Dänzer]
* Use BO's location rather than the AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED
  flag to determine whether to account a move for visible VRAM in most
  cases.
* Use a single

if (adev->mc.visible_vram_size < adev->mc.real_vram_size) {

  block in amdgpu_cs_get_threshold_for_moves.

Fixes: 95844d20ae02 (drm/amdgpu: throttle buffer migrations at CS using a fixed MBps limit (v2))
Signed-off-by: John Brooks <john@fastquake.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Add vis_vramlimit module parameter

Allow specifying a limit on visible VRAM via a module parameter. This is
helpful for testing performance under visible VRAM pressure.

v2: Add cast to 64-bit (Christian König)

Signed-off-by: John Brooks <john@fastquake.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: change gartsize default to 256MB

Limit the default GART size and save a lot of VRAM.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add new gttsize module parameter v2

This allows setting the gtt size independent of the gart size.

v2: fix copy and paste typo

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: limit the GTT manager address space

We should only cover the GART size with the GTT manager.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: consistent name all GART related parts

Rename symbols from gtt_ to gart_ as appropriate.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove gtt_base_align handling

Not used any more.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: move GART struct and function into amdgpu_gart.h v2

No functional change, just cleanup.

v2: rebased, keep gart name.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: check scratch registers to see if we need post (v2)

Rather than checking the CONGIG_MEMSIZE register as that may
not be reliable on some APUs.

v2: The scratch register is only used on CIK+

Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/soc15: init nbio registers for vega10

Call nbio init registers on hw_init to set up any
nbio registers that need initialization at hw init time.

Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add nbio 6.1 register init function

Used for nbio registers that need to be initialized.  Currently
only used for a golden setting that got missed on some boards.

Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: added didt support for vega10

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: added grbm_idx_mutex lock/unlock to cgs v2

  - v2: rename param 'en' as 'lock'

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: added support for new se_cac_idx APIs to cgs

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: added soc15 support for new se_cac_idx APIs

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: added new se_cac_idx r/w APIs v2

  - v2: added missing spinlock init

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: added index gc cac read/write apis for vega10

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use TTM values instead of MC values for the info queries

Use the TTM values instead of the hardware config here.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove maximum BO size limitation v2

We can finally remove this now.

v2: remove now unused max_size variable as well.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: stop mapping BOs to GTT

No need to map BOs to GTT on eviction and intermediate transfers any more.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use the GTT windows for BO moves v2

This way we don't need to map the full BO at a time any more.

v2: use fixed windows for src/dst

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add amdgpu_gart_map function v2

This allows us to write the mapped PTEs into
an IB instead of the table directly.

v2: fix build with debugfs enabled, remove unused assignment

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: reserve the first 2x512 pages of GART

We want to use them as remap address space.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: make arrays pctl0_data and pctl1_data static

The arrays pctl0_data and pctl1_data do not need to be in global scope,
so them both static.

Cleans up sparse warnings:
symbol 'pctl0_data' was not declared. Should it be static?
symbol 'pctl1_data' was not declared. Should it be static?

Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gmc9: get vram width from atom for Raven

Get it from the system info table.

Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/atomfirmware: implement vram_width for APUs

Implement support using the new atomfirmware system info table.

Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/atom: fix atom_fw check

Not all vbios images seem to set the version appropriately.
Switch the check based on asic type instead.

Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Free resources of bo_list when idr_alloc fails

Signed-off-by: Alex Xie <AlexBin.Xie@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian König<christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add avfs check for old asics on Vi.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: move VI common AVFS code to smu7_smumgr.c

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: refine avfs enable code on fiji.

1. simplify avfs state switch.
2. delete save/restore VFT table functions as not support
   by fiji.
3. implement thermal_avfs_enable funciton.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: fix avfs state update error on polaris.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: fixed wrong data type declaration for ppfeaturemask

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: set firmware loading type as direct by default for raven

In previous case, driver can't enable psp via the kernel parameter for raven.
We should open this path and set it as direct by default till psp firmware
loading is workable.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: make psp cmd buffer as a reserve memory

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix missed asd bo free when hw_fini

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove superfluous check

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: NO KIQ usage on nbio hdp flush routine

nbio hdp flush routine are called within atomic context.
Avoid use KIQ when write to the HDP_MEM_COHERENCY_FLUSH_CNTL register
since this register has its own VF copy

Signed-off-by: Shaoyun Liu <Shaoyun.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Add WREG32_SOC15_NO_KIQ macro define

Signed-off-by: Shaoyun Liu <Shaoyun.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:fix world switch hang

for SR-IOV, we must keep the pipeline-sync in the protection
of COND_EXEC, otherwise the command consumed by CPG is not
consistent when world switch triggerd, e.g.:

world switch hit and the IB frame is skipped so the fence
won't signal, thus CP will jump to the next DMAframe's pipeline-sync
command, and it will make CP hang foever.

after pipelin-sync moved into COND_EXEC the consistency can be
guaranteed

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: enable ACG feature on vega10.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add acg support in pptable for vega10

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: export ACG related smu message for vega10

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add avfs profiling_info_v4_2 support on Vega10.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add ACG SMU firmware for other vega10 variants

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: drop SMU_DRIVER_IF_VERSION check for some vega10 variants

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add workaround for S3 issues on some vega10 boards

Certain MC registers need a delay after writing them to properly
update in the init sequence.

Signed-off-by: Ken Wang <Ken.Wang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/atombios: add function for whether we need asic_init

Check the atom scratch registers to see if asic_init is complete
or not.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: unify some atombios/atomfirmware scratch reg functions

Now that we use a pointer to the scratch reg start offset,
most of the functions were duplicated.

Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/atombios: use bios_scratch_reg_offset for atombios

Align with the atomfirmware code.

Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm: amd: amdgpu: constify ttm_place structures.

ttm_place are not supposed to change at runtime. All functions
working with ttm_place provided by <drm/ttm/ttm_placement.h> work
with const ttm_place. So mark the non-const structs as const.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Arvind Yadav <arvind.yadav.cs@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm: radeon: constify drm_prop_enum_list structures.

drm_prop_enum_lists are not supposed to change at runtime. All functions
working with drm_prop_enum_list provided by <drm/drm_property.h> work with
const drm_prop_enum_list. So mark the non-const structs as const.

File size before:
   text    data     bss     dec     hex filename
  18276     384       0   18660    48e4 drivers/gpu/drm/radeon/radeon_display.o

File size After adding 'const':
   text    data     bss     dec     hex filename
  18660       0       0   18660    48e4 drivers/gpu/drm/radeon/radeon_display.o

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Arvind Yadav <arvind.yadav.cs@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm: radeon: radeon_ttm: constify ttm_place structures.

ttm_place are not supposed to change at runtime. All functions
working with ttm_place provided by <drm/ttm/ttm_placement.h> work
with const ttm_place. So mark the non-const structs as const.

File size before:
   text    data     bss     dec     hex filename
   9235     344     136    9715    25f3 drivers/gpu/drm/radeon/radeon_ttm.o

File size After adding 'const':
   text    data     bss     dec     hex filename
   9267     312     136    9715    25f3 drivers/gpu/drm/radeon/radeon_ttm.o

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Arvind Yadav <arvind.yadav.cs@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: trace VM flags as 64bits

Otherwise the upper bits are lost.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Xie <AlexBin.Xie@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove stale TODO comment

That is already fixed.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/sched: print sched job id in amd_sched_job trace

This makes it easier to correlate amd_sched_job with with other trace
points that don't log the job pointer.

v2: don't print the sched_job pointer (Andres)

Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Andres Rodriguez <andresx7@gmail.com>
Reviewed-by: Christian König <christian.koenig@amd.com>

drm/amdgpu: update pctl1 ram index/data for mmhub on raven

Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>

drm/amdgpu: add check when no firmware need to load

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: bind BOs with GTT space allocated directly v2

This avoids binding them later on.

v2: fix typo in function name

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>

drm/amdgpu: bind BOs to TTM only once

No need to do this on every round.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>

drm/amdgpu: add vm_needs_flush parameter to amdgpu_copy_buffer

This allows us to flush the system VM here.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>

drm/amdgpu: allow flushing VMID0 before IB execution as well

This allows us to queue IBs which needs an up to date system domain as well.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>

drm/amdgpu: fix amdgpu_ring_write_multiple

Overwriting still used ring content has a low probability to cause
problems, not writing at all has 100% probability to cause problems.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>

drm/amdgpu: move ring helpers to amdgpu_ring.h

Keep them where they belong.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>

drm/radeon: add header comment for clarification to vce_v2_0_enable_mgcg()

Add function header comment to make it clear that local variable sw_cg
is used for debugging and it should not be removed.

Addresses-Coverity-ID: 1198635
Cc: Alex Deucher <alexdeucher@gmail.com>
Signed-off-by: Gustavo A. R. Silva <garsilva@embeddedor.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Update default vram_page_split description

This was updated to 512 and the description update got missed

Signed-off-by: Kent Russell <kent.russell@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Changed CU reservation golden settings

With previous golden settings, compute task can't use
reserved LDS (32K) on CU0 and CU1. On 64K LDS system,
if compute work group allocate more than 32K LDS, then
it can't be dispatched to CU0 and CU1 because of the
reservation. This enables compute task to use reserved
LDS on CU0 and CU1.

Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix amdgpu_debugfs_gem_bo_info

Otherwise we trigger a bunch of WARN_ONs when this is called.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: cleanup initializing gtt_size

Stop spreading the code over all GMC generations.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Support passing amdgpu critical error to host via GPU Mailbox.

This feature works for SRIOV enviroment. For non-SRIOV enviroment, the
trans_error function does nothing.

The error information includes error_code (16bit), error_flags(16bit)
and error_data(64bit). Since there are not many errors, we keep the
errors in an array and transfer all errors to Host before amdgpu
initialization function (amdgpu_device_init) exit.

Signed-off-by: Gavin Wan <Gavin.Wan@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Allow vblank_disable_immediate.

With instantaneous high precision vblank timestamping
that updates at leading edge of vblank, a cooked hw
vblank counter which increments at leading edge of
vblank, and reliable page flip execution and completion
at leading edge of vblank, we should meet the requirements
for fast/immediate vblank irq disable/enable.

Testing on Linux-4.12-rc5 + drm-next on a Radeon R9 380
Tonga Pro (DCE 10) with timing measurement equipment
indicates this works fine, so allow immediate vblank
disable for power saving.

For debugging in case of unexpected trouble, booting
with kernel cmdline option drm.vblankoffdelay=0
(or echo 0 > /sys/module/drm/parameters/vblankoffdelay)
would keep vblank irqs permanently on to approximate old
behavior.

Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Mario Kleiner <mario.kleiner.de@gmail.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Allow vblank_disable_immediate.

With instantaneous high precision vblank timestamping
that updates at leading edge of vblank, a cooked hw
vblank counter which increments at leading edge of
vblank, and reliable page flip execution and completion
at leading edge of vblank, we should meet the requirements
for fast/immediate vblank irq disable/enable.

Testing on Linux-4.12-rc5 + drm-next on a Radeon HD 5770
(DCE 4) with timing measurement equipment indicates this
works fine, so allow immediate vblank disable for power
saving.

For debugging in case of unexpected trouble, booting
with kernel cmdline option drm.vblankoffdelay=0
(or echo 0 > /sys/module/drm/parameters/vblankoffdelay)
would keep vblank irqs permanently on to approximate old
behavior.

Reviewed-and-Tested-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Mario Kleiner <mario.kleiner.de@gmail.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove *_mc_access from display funcs

These are no longer needed now that we use the fb_location
programmed by the vbios.

Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: drop set_vga_render_state from display funcs

Not used.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gmc6: drop fb location programming

No need to do this as the vbios does this for us.  As such
we no longer need to stop the mc during init.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gmc7: drop fb location programming

No need to do this as the vbios does this for us.  As such
we no longer need to stop the mc during init.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gmc8: drop fb location programming

No need to do this as the vbios does this for us.  As such
we no longer need to stop the mc during init.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gmc6: use the vram location programmed by the vbios

This makes mc programming much simpler in future patches.

Since evergreen, the vbios has been programming the fb location
to the proper vram size.  The only reason to reprogram it would
be to change the location.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gmc7: use the vram location programmed by the vbios

This makes mc programming much simpler in future patches.

Since evergreen, the vbios has been programming the fb location
to the proper vram size.  The only reason to reprogram it would
be to change the location.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gmc8: use the vram location programmed by the vbios

This makes mc programming much simpler in future patches.

Since evergreen, the vbios has been programming the fb location
to the proper vram size.  The only reason to reprogram it would
be to change the location.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: disable vga render in dce hw_init

This got dropped accidently with the fb location changes, but for
some reason, this doesn't seem to cause an issue on all cards which
is why I never saw it despite extensive testing.  I suspect it may
only be an issue on systems with a legacy sbios that enables vga.

Tested-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: simplify VM shadow handling v2

Now that we don't join PTE updates any more we don't need to call
the update function twice for this.

v2: rebased

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: enable 4 level page table on raven (v3)

v1: enable 4 level-page table on raven
v2: add back legacy 2 level page table on raven
v3: set num_level in initial switch statement

Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use kernel is_power_of_2 rather than local version

Use the kernel provided version.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Merge branch 'drm-next-4.13' of git://people.freedesktop.org/~agd5f/linux into drm-next

single r700 fix.
* 'drm-next-4.13' of git://people.freedesktop.org/~agd5f/linux:
  drm/radeon: Fix eDP for single-display iMac10,1 (v2)

Merge tag 'drm-misc-next-fixes-2017-07-10' of git://anongit.freedesktop.org/git/drm-misc into drm-next

Core Changes:
- Fix empty timestamps on hw without vlbank counter (Laurent)
- Clear atomic state before retrying ww/mutex acquisition in remove_fb (Maarten)

Driver Changes:
- rockchip: Fix incorrect NULL pointer check after allocation (Gustavo)

Cc: Gustavo A. R. Silva <garsilva@embeddedor.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
* tag 'drm-misc-next-fixes-2017-07-10' of git://anongit.freedesktop.org/git/drm-misc:
  drm/rockchip: fix NULL check on devm_kzalloc() return value
  drm/atomic: Add missing drm_atomic_state_clear to atomic_remove_fb
  drm: vblank: Fix vblank timestamp update
  DRM: Fix an incorrectly formatted table
  bridge: Fix panel-bridge error return on !panel.
  drm/rockchip: gem: add the lacks lock and trivial changes

Merge tag 'drm-intel-next-fixes-2017-07-11' of git://anongit.freedesktop.org/git/drm-intel into drm-next

drm/i915 fixes for v4.13-rc1

* tag 'drm-intel-next-fixes-2017-07-11' of git://anongit.freedesktop.org/git/drm-intel:
  drm/i915: Make DP-MST connector info work
  drm/i915/gvt: Use fence error from GVT request for workload status
  drm/i915/gvt: remove scheduler_mutex in per-engine workload_thread
  drm/i915/gvt: Revert "drm/i915/gvt: Fix possible recursive locking issue"
  drm/i915/gvt: Audit the command buffer address
  drm/i915/gvt: Fix a memory leak in intel_gvt_init_gtt()
  drm/i915/fbdev: Check for existence of ifbdev->vma before operations
  drm/i915: Hold RPM wakelock while initializing OA buffer
  drm/i915/cnl: Fix the CURSOR_COEFF_MASK used in DDI Vswing Programming
  drm/i915/cfl: Fix Workarounds.
  drm/i915: Avoid undefined behaviour of "u32 >> 32"
  drm/i915: reintroduce VLV/CHV PFI programming power domain workaround
  drm/i915: Fix an error checking test
  drm/i915: Disable MSI for all pre-gen5
  drm/i915/gvt: Make function dpy_reg_mmio_readx safe
  drm/i915/gvt: Don't read ADPA_CRT_HOTPLUG_MONITOR from host
  drm/i915/gvt: Set initial PORT_CLK_SEL vreg for BDW
  drm/i915/gvt: Fix inconsistent locks holding sequence
  drm/i915/gvt: Fix possible recursive locking issue

Merge branch 'mediatek-drm-next-4.13' of https://github.com/ckhu-mediatek/linux.git-tags into drm-next

This include new color format support and some fixups.

* 'mediatek-drm-next-4.13' of https://github.com/ckhu-mediatek/linux.git-tags:
  drm/mediatek: separate color module to fixup error memory reallocation
  drm/mediatek: check for memory allocation failure
  drm/mediatek: re-phrase DRM_INFO error message
  drm/mediatek: use platform_register_drivers
  drm/mediatek: Support UYVY and YUYV format for overlay

Merge tag 'gvt-fixes-2017-07-11' of https://github.com/01org/gvt-linux into drm-intel-next-fixes

gvt-fixes-2017-07-11

- Revert "drm/i915/gvt: Fix possible recursive locking issue" (Chuanxiao),
  which is incomplete fix and it's actually VFIO issue, so revert.
- remove unneeded scheduler mutex for performance fix (Weinan)
- other misc error handling fix and cmd address audit

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170711055333.jhrmvx6ilvg2qlnn@zhen-hp.sh.intel.com

drm/i915: Make DP-MST connector info work

Commit 9a148a96fc3a ("drm/i915/debugfs: add dp mst info") adds support
for DP-MST to intel_connector_info, but forgot to remove the early
return for DP-MST.

Remove it, and print out MST connectors directly.

Fixes: 9a148a96fc3a ("drm/i915/debugfs: add dp mst info")
Cc: <stable@vger.kernel.org> # v4.11+
Cc: Dhinakaran Pandiyan <dhinakaran.pandiyan@intel.com>
Cc: Libin Yang <libin.yang@intel.com>
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170626083349.24389-1-maarten.lankhorst@linux.intel.com
Reviewed-by: Dhinakaran Pandiyan <dhinakaran.pandiyan@intel.com>
(cherry picked from commit 77d1f615c78a73a04254fa2bff07ee9fa27145d9)
Signed-off-by: Jani Nikula <jani.nikula@intel.com>

drm/i915/gvt: Use fence error from GVT request for workload status

The req->fence.error will be set if this request caused GPU hang so
we can use this value to workload->status to indicate whether this
GVT request caused any problem. If it caused GPU hang, we shouldn't
trigger any context switch back to the guest.

v2:
- only take -EIO from fence->error. (Zhenyu)

Fixes: 8f1117abb408 (drm/i915/gvt: handle workload lifecycle properly)
Signed-off-by: Chuanxiao Dong <chuanxiao.dong@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

drm/i915/gvt: remove scheduler_mutex in per-engine workload_thread

For the vGPU workloads, now GVT-g use per vGPU scheduler, the per-ring
work_thread only pick workload belongs to the current vGPU. And with time
slice based scheduler, it waits all the engines become idle before do vGPU
switch. So we can run free dispatch in per-ring work_thread, different ring
running in different 'vGPU' won't happen.

For the workloads between vGPU and Host, this scheduler_mutex can't block
host to dispatch workload into other ring engines.

Here remove this mutex since it impacts the performance when applications
use more than 1 ring engines in 1 vgpu.

ring0 running in vGPU1, ring1 running in Host. Will happen.
ring0 running in vGPU1, ring1 running in vGPU2. Won't happen.

Signed-off-by: Weinan Li <weinan.z.li@intel.com>
Signed-off-by: Ping Gao <ping.a.gao@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

drm/i915/gvt: Revert "drm/i915/gvt: Fix possible recursive locking issue"

This reverts commit 62d02fd1f807bf5a259a242c483c9fb98a242630.

The rwsem recursive trace should not be fixed from kvmgt side by using
a workqueue and it is an issue should be fixed in VFIO. So this one
should be reverted.

Signed-off-by: Chuanxiao Dong <chuanxiao.dong@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: stable@vger.kernel.org # v4.10+
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

drm/i915/gvt: Audit the command buffer address

The command buffer address in context like ring buffer base address
and wa_ctx address need to be audit to make sure they are in the
valid GGTT range.

Signed-off-by: Ping Gao <ping.a.gao@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>