skip to Main Content

I am trying to figure out how to form my database query to build up a collection of projects, each of which has a set of personnel with their assigned working periods and the daily shift hours that they submit.

In the code I have so far, I select the projects that are running for the current month. For each of those I select the personnel that are working on the project during that month. Then for each of the personnel I make another 2 queries to get their assignment dates and submitted shift details.

This does work, and creates the data structure I want, but it’s slow due to the large number of individual database queries.

There must be a more efficient way to do this through subqueries or something?

$projects = Project::select([
    'projects.id',
    'projects.name',
    'projects.start',
    'projects.finish'
])
->where('projects.start', '<=', Carbon::now()->endOfMonth())
->where('projects.finish', '>=', Carbon::now()->startOfMonth())
->get();

foreach($projects as $project)
{
    $project->personnel = ProjectUser::select([
        'project_users.id',
        'project_users.user_id',
        'users.name',
        'project_users.role'
    ])
    ->join('users', 'users.id', '=', 'project_users.user_id')
    ->join('assignments', 'assignments.project_user_id', '=', 'project_users.id')
    ->where('project_users.project_id', $project->id)
    ->where('assignments.start', '<=', Carbon::now()->endOfMonth())
    ->where('assignments.finish', '>=', Carbon::now()->startOfMonth())
    ->distinct('users.name')
    ->get();

    foreach($project->personnel as $person)
    {
        $person->assignments = Assignment::select([
            'assignments.start',
            'assignments.finish',
            'assignments.travel_out',
            'assignments.travel_home'
        ])
        ->where('assignments.project_user_id', $person->project_user_id)
        ->get();

        $person->shifts = WorkShift::select([
            'work_shifts.id',
            'work_shifts.role',
            'work_shifts.date',
            'work_shifts.start',
            'work_shifts.hours',
            'work_shifts.status',
            'work_shifts.remarks'
        ])
        ->where('work_shifts.user_id', $person->user_id)
        ->where('work_shifts.project_id', $project->id)
        ->get();
    }
}

Update

I now have this mostly working through the use of model relationships. My last problem is that I can filter the projects within a date range at the top level in my controller, but I also need to apply that filter to the assignments and work_shifts.

So, I only want to retrieve project_users that have assignments within the requested date range and then pick out only the assignments and work_shifts that match.

What I have so far is…

// Controller
public function load(Request $request)
{
  $projects = Project::select([
    Project::ID,
    Project::NAME,
    Project::START,
    Project::FINISH
  ])
    // Only able to filter by date range here?!?
    ->where(Project::START, '<=', Carbon::now()->endOfYear())
    ->where(Project::FINISH, '>=', Carbon::now()->startOfYear())
    ->with('project_staff')
    ->orderBy(Project::START)
    ->get();

  return response()->json([
    'projects' => $projects
  ]);
}

// Project Model
// Just want to get the staff that are assigned to the project
// between the selected date range
public function project_staff()
{
  return $this->hasMany(ProjectUser::class)
    ->select([
      ProjectUser::ID,
      ProjectUser::PROJECT_ID,
      ProjectUser::USER_ID,
      User::NAME,
      ProjectUser::ROLE
    ])
    ->whereIn(ProjectUser::STATUS, [
      ProjectUser::STATUS_RESERVED,
      ProjectUser::STATUS_ASSIGNED,
      ProjectUser::STATUS_ACCEPTED
    ])
    ->join(User::TABLE, User::ID, '=', ProjectUser::USER_ID)
    ->with([
      'assignments',
      'shifts'
    ]);
}

// Assignments Model
// Again, just want the assignments and workshifts that fall
// within the selected date range
public function assignments()
{
  return $this->hasMany(Assignment::class, 'projectUser_id')
    ->select([
      Assignment::PROJECT_USER_ID,
      Assignment::START,
      Assignment::FINISH,
      Assignment::TRAVEL_OUT,
      Assignment::TRAVEL_HOME
    ]);
}

public function shifts()
{
  return $this->hasMany(WorkShift::class, ['project_id','user_id'], ['project_id','user_id'])
    ->select([
      WorkShift::ID,
      WorkShift::USER_ID,
      WorkShift::PROJECT_ID,
      WorkShift::ROLE,
      WorkShift::DATE,
      WorkShift::START,
      WorkShift::HOURS,
      WorkShift::STATUS,
      WorkShift::REMARKS
    ]);
}

// WorkShift Model
public function projectUser()
{
    return $this->belongsTo(ProjectUser::class, ['project_id','user_id'], ['project_id','user_id']);
}

2

Answers


  1. Chosen as BEST ANSWER

    After a day of much learning, the solution I've found is to have very simple relationships between the models and implement all the additional query logic in filter functions attached to the relationship query when it's called using with().

    See https://laravel.com/docs/8.x/eloquent-relationships#constraining-eager-loads

    My code is now working far more efficiently, making only 4 calls to the database in about 10-20ms in my dev-env, and structuring the results data in the correct form to simply return the whole collection as a JSON response.

    // Controller
    public function load(Request $request)
    {
      $start = Carbon::parse($request->input('start_date'));
      $end = Carbon::parse($request->input('end_date'));
    
      $projects = Project::select([
        Project::ID,
        Project::NAME,
        Project::START,
        Project::FINISH
      ])
        ->where(Project::START, '<=', $end)
        ->where(Project::FINISH, '>=', $start)
        ->with([
          // Filter results of the model relationship, passing $start and
          // $end dates down to propagate through to the nested layers
          'project_staff' => $this->filter_project_staff($start, $end)
        ])
        ->orderBy(Project::START)
        ->get();
    
        return response()->json([
          'projects' => $projects
        ]);
    }
    
    private function filter_project_staff($start, $end)
    {
      return function ($query) use ($start, $end) {
        $query
          ->select([
            ProjectUser::ID,
            ProjectUser::PROJECT_ID,
            ProjectUser::USER_ID,
            User::NAME,
            ProjectUser::ROLE
          ])
          ->whereIn(ProjectUser::STATUS, [
            ProjectUser::STATUS_RESERVED,
            ProjectUser::STATUS_ASSIGNED,
            ProjectUser::STATUS_ACCEPTED
          ])
          ->join(User::TABLE, User::ID, '=', ProjectUser::USER_ID)
          ->join(Assignment::TABLE, Assignment::PROJECT_USER_ID, '=', ProjectUser::ID)
          ->where(Assignment::START, '<=', $end)
          ->where(Assignment::FINISH, '>=', $start)
          ->distinct(User::NAME)
          ->orderBy(ProjectUser::ROLE, 'desc')
          ->orderBy(User::NAME)
          ->with([
            // Filter results of the nested model relationships
            'assignments' => $this->filter_assignments($start, $end),
            'shifts' => $this->filter_shifts($start, $end)
          ]);
      };
    }
    
    private function filter_assignments($start, $end)
    {
      return function ($query) use ($start, $end) {
        $query->select([
          Assignment::PROJECT_USER_ID,
          Assignment::START,
          Assignment::FINISH,
          Assignment::TRAVEL_OUT,
          Assignment::TRAVEL_HOME
        ])
          ->where(Assignment::START, '<=', $end)
          ->where(Assignment::FINISH, '>=', $start)
          ->orderBy(Assignment::START);
      };
    }
    
    private function filter_shifts($start, $end)
    {
      return function ($query) use ($start, $end) {
        $query->select([
          WorkShift::ID,
          WorkShift::USER_ID,
          WorkShift::PROJECT_ID,
          WorkShift::ROLE,
          WorkShift::DATE,
          WorkShift::START,
          WorkShift::HOURS,
          WorkShift::STATUS,
          WorkShift::REMARKS
        ])
          ->where(WorkShift::DATE, '>=', $start)
          ->where(WorkShift::DATE, '<=', $end)
          ->orderBy(WorkShift::DATE);
      };
    }
    
    ///////////////
    //Project Model
    public function project_staff(): HasMany
    {
        return $this->hasMany(ProjectUser::class);
    }
    
    ////////////////////
    // ProjectUser Model
    public function project(): BelongsTo
    {
        return $this->belongsTo(Project::class);
    }
    
    public function assignments(): HasMany
    {
        return $this->hasMany(Assignment::class, 'projectUser_id');
    }
    
    public function shifts()
    {
        // Using awobaz/compoships to simplify creating the relationship using a composite key
        return $this->hasMany(WorkShift::class, ['project_id','user_id'], ['project_id','user_id']);
    }
    
    ///////////////////
    // Assignment Model
    public function project_user(): BelongsTo
    {
        return $this->belongsTo(ProjectUser::class, 'projectUser_id');
    }
    
    
    // WorkShift Model
    public function project_user()
    {
        return $this->belongsTo(ProjectUser::class, ['project_id','user_id'], ['project_id','user_id']);
    }
    

    Many thanks to anyone that put time and thought into helping me with this problem. Your effort is very much appreciated.


  2. As mentioned before, you can use Laravel built in relationships, or, you can construct a giant query and join all the data together;

    Like so:

    <?php
        
    $projectUsers = ProjectUser::select([
            'project_users.id', // define aliases for all columns
            'project_users.user_id',
            'users.name',
            'project_users.role',
            'projects.id',
            'projects.name',
            'projects.start',
            'projects.finish',
            'assignments.start',
            'assignments.finish',
            'assignments.travel_out',
            'assignments.travel_home',
            'work_shifts.id',
            'work_shifts.role',
            'work_shifts.date',
            'work_shifts.start',
            'work_shifts.hours',
            'work_shifts.status',
            'work_shifts.remarks'
        ])
        ->join('users', 'users.id', 'project_users.user_id'
        ->leftJoin('assignments', 'assignment.project_user_id', 'project_users.id')
        ->leftJoin('projects', fn ($query) => $query
            ->where('projects.start', '<=', Carbon::now()->endOfMonth())
            ->where('projects.finish', '>=', Carbon::now()->startOfMonth())
        )
        ->leftJoin('work_shifts', fn ($query) => $query
            ->whereColumn('work_shifts.id', 'project_users.id')
            ->whereColumn('work_shifts.project_id', 'projects.id')
        )
        ->whereColumn('project_users.project_id', 'projects.id')
        ->where('assignments.start', '<=', Carbon::now()->endOfMonth())
        ->where('assignments.finish', '>=', Carbon::now()->startOfMonth())
        ->distinct('users.name')
        ->cursor();
    
    foreach ($projectUsers as $user) {
        // ...
    }
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search